{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bagging, Random Forests and Boosting for Regression\n",
    "### Predicting Diamond Prices with"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Set Information:\n",
    "\n",
    "**A dataset containing the prices and other features of almost 54,000 diamonds.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Features description\n",
    "\n",
    "**Number of Attributes: 10 (9 predictive features, 1 target)**\n",
    "\n",
    "Feature Information: A data frame with 53,940 rows and 10 variables:\n",
    "\n",
    "price: price in US dollars (\\$326--\\$18,823) **(target)**\n",
    "\n",
    "carat: weight of the diamond (0.2--5.01)\n",
    "\n",
    "cut: quality of the cut (Fair, Good, Very Good, Premium, Ideal)\n",
    "\n",
    "color: diamond colour, from J (worst) to D (best)\n",
    "\n",
    "clarity: a measurement of how clear the diamond is (I1 (worst), SI2, SI1, VS2, VS1, VVS2, VVS1, IF (best))\n",
    "\n",
    "x: length in mm (0--10.74)\n",
    "\n",
    "y: width in mm (0--58.9)\n",
    "\n",
    "z: depth in mm (0--31.8)\n",
    "\n",
    "depth: total depth percentage = z / mean(x, y) = 2 * z / (x + y) (43--79)\n",
    "\n",
    "table: width of top of diamond relative to widest point (43--95)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Importing useful libraries\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.23</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.5</td>\n",
       "      <td>55.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.21</td>\n",
       "      <td>Premium</td>\n",
       "      <td>E</td>\n",
       "      <td>SI1</td>\n",
       "      <td>59.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3.84</td>\n",
       "      <td>2.31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.23</td>\n",
       "      <td>Good</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>56.9</td>\n",
       "      <td>65.0</td>\n",
       "      <td>327</td>\n",
       "      <td>4.05</td>\n",
       "      <td>4.07</td>\n",
       "      <td>2.31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.29</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.4</td>\n",
       "      <td>58.0</td>\n",
       "      <td>334</td>\n",
       "      <td>4.20</td>\n",
       "      <td>4.23</td>\n",
       "      <td>2.63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.31</td>\n",
       "      <td>Good</td>\n",
       "      <td>J</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>335</td>\n",
       "      <td>4.34</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.24</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>J</td>\n",
       "      <td>VVS2</td>\n",
       "      <td>62.8</td>\n",
       "      <td>57.0</td>\n",
       "      <td>336</td>\n",
       "      <td>3.94</td>\n",
       "      <td>3.96</td>\n",
       "      <td>2.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.24</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>I</td>\n",
       "      <td>VVS1</td>\n",
       "      <td>62.3</td>\n",
       "      <td>57.0</td>\n",
       "      <td>336</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.26</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>H</td>\n",
       "      <td>SI1</td>\n",
       "      <td>61.9</td>\n",
       "      <td>55.0</td>\n",
       "      <td>337</td>\n",
       "      <td>4.07</td>\n",
       "      <td>4.11</td>\n",
       "      <td>2.53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.22</td>\n",
       "      <td>Fair</td>\n",
       "      <td>E</td>\n",
       "      <td>VS2</td>\n",
       "      <td>65.1</td>\n",
       "      <td>61.0</td>\n",
       "      <td>337</td>\n",
       "      <td>3.87</td>\n",
       "      <td>3.78</td>\n",
       "      <td>2.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.23</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>H</td>\n",
       "      <td>VS1</td>\n",
       "      <td>59.4</td>\n",
       "      <td>61.0</td>\n",
       "      <td>338</td>\n",
       "      <td>4.00</td>\n",
       "      <td>4.05</td>\n",
       "      <td>2.39</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   carat        cut color clarity  depth  table  price     x     y     z\n",
       "0   0.23      Ideal     E     SI2   61.5   55.0    326  3.95  3.98  2.43\n",
       "1   0.21    Premium     E     SI1   59.8   61.0    326  3.89  3.84  2.31\n",
       "2   0.23       Good     E     VS1   56.9   65.0    327  4.05  4.07  2.31\n",
       "3   0.29    Premium     I     VS2   62.4   58.0    334  4.20  4.23  2.63\n",
       "4   0.31       Good     J     SI2   63.3   58.0    335  4.34  4.35  2.75\n",
       "5   0.24  Very Good     J    VVS2   62.8   57.0    336  3.94  3.96  2.48\n",
       "6   0.24  Very Good     I    VVS1   62.3   57.0    336  3.95  3.98  2.47\n",
       "7   0.26  Very Good     H     SI1   61.9   55.0    337  4.07  4.11  2.53\n",
       "8   0.22       Fair     E     VS2   65.1   61.0    337  3.87  3.78  2.49\n",
       "9   0.23  Very Good     H     VS1   59.4   61.0    338  4.00  4.05  2.39"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# importing data\n",
    "data_path= '../data/diamonds.csv'\n",
    "diamonds = pd.read_csv(data_path)\n",
    "diamonds.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(53940, 10)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using one-hot encoding for categorical features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the transformations we must perform is to tranform the categorical features to the one-hot-encoding format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Ideal' 'Premium' 'Good' 'Very Good' 'Fair']\n",
      "['E' 'I' 'J' 'H' 'F' 'G' 'D']\n",
      "['SI2' 'SI1' 'VS1' 'VS2' 'VVS2' 'VVS1' 'I1' 'IF']\n"
     ]
    }
   ],
   "source": [
    "print(diamonds['cut'].unique())\n",
    "print(diamonds['color'].unique())\n",
    "print(diamonds['clarity'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "diamonds = pd.concat([diamonds, pd.get_dummies(diamonds['cut'], prefix='cut', drop_first=True)],axis=1)\n",
    "diamonds = pd.concat([diamonds, pd.get_dummies(diamonds['color'], prefix='color', drop_first=True)],axis=1)\n",
    "diamonds = pd.concat([diamonds, pd.get_dummies(diamonds['clarity'], prefix='clarity', drop_first=True)],axis=1)\n",
    "diamonds.drop(['cut','color','clarity'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>carat</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>cut_Good</th>\n",
       "      <th>cut_Ideal</th>\n",
       "      <th>cut_Premium</th>\n",
       "      <th>...</th>\n",
       "      <th>color_H</th>\n",
       "      <th>color_I</th>\n",
       "      <th>color_J</th>\n",
       "      <th>clarity_IF</th>\n",
       "      <th>clarity_SI1</th>\n",
       "      <th>clarity_SI2</th>\n",
       "      <th>clarity_VS1</th>\n",
       "      <th>clarity_VS2</th>\n",
       "      <th>clarity_VVS1</th>\n",
       "      <th>clarity_VVS2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.23</td>\n",
       "      <td>61.5</td>\n",
       "      <td>55.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.43</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.21</td>\n",
       "      <td>59.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3.84</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.23</td>\n",
       "      <td>56.9</td>\n",
       "      <td>65.0</td>\n",
       "      <td>327</td>\n",
       "      <td>4.05</td>\n",
       "      <td>4.07</td>\n",
       "      <td>2.31</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.29</td>\n",
       "      <td>62.4</td>\n",
       "      <td>58.0</td>\n",
       "      <td>334</td>\n",
       "      <td>4.20</td>\n",
       "      <td>4.23</td>\n",
       "      <td>2.63</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.31</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>335</td>\n",
       "      <td>4.34</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.75</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   carat  depth  table  price     x     y     z  cut_Good  cut_Ideal  \\\n",
       "0   0.23   61.5   55.0    326  3.95  3.98  2.43         0          1   \n",
       "1   0.21   59.8   61.0    326  3.89  3.84  2.31         0          0   \n",
       "2   0.23   56.9   65.0    327  4.05  4.07  2.31         1          0   \n",
       "3   0.29   62.4   58.0    334  4.20  4.23  2.63         0          0   \n",
       "4   0.31   63.3   58.0    335  4.34  4.35  2.75         1          0   \n",
       "\n",
       "   cut_Premium      ...       color_H  color_I  color_J  clarity_IF  \\\n",
       "0            0      ...             0        0        0           0   \n",
       "1            1      ...             0        0        0           0   \n",
       "2            0      ...             0        0        0           0   \n",
       "3            1      ...             0        1        0           0   \n",
       "4            0      ...             0        0        1           0   \n",
       "\n",
       "   clarity_SI1  clarity_SI2  clarity_VS1  clarity_VS2  clarity_VVS1  \\\n",
       "0            0            1            0            0             0   \n",
       "1            1            0            0            0             0   \n",
       "2            0            0            1            0             0   \n",
       "3            0            0            0            1             0   \n",
       "4            0            1            0            0             0   \n",
       "\n",
       "   clarity_VVS2  \n",
       "0             0  \n",
       "1             0  \n",
       "2             0  \n",
       "3             0  \n",
       "4             0  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting the train and test sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.preprocessing import RobustScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "target_name = 'price'\n",
    "robust_scaler = RobustScaler()\n",
    "X = diamonds.drop('price', axis=1)\n",
    "feature_names = X.columns\n",
    "X = robust_scaler.fit_transform(X)\n",
    "y = diamonds[target_name]\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=55)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing a DataFrame for model analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "models = pd.DataFrame(index=['train_mse', 'test_mse'], \n",
    "                      columns=['KNN', 'Bagging', 'RandomForest', 'Boosting'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A. K-Nearest Neighbors Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 1. Import the estimator object (model)\n",
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "# 2. Create an instance of the estimator\n",
    "knn = KNeighborsRegressor(n_neighbors=20, weights='distance', metric='euclidean', n_jobs=-1)\n",
    "# 3. Use the trainning data to train the estimator\n",
    "knn.fit(X_train, y_train)\n",
    "# 4. Evaluate the model\n",
    "models.loc['train_mse','KNN'] = mean_squared_error(y_pred=knn.predict(X_train), \n",
    "                                                    y_true=y_train)\n",
    "\n",
    "models.loc['test_mse','KNN'] = mean_squared_error(y_pred=knn.predict(X_test), \n",
    "                                                   y_true=y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## B. Bagging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 1. Import the estimator object (model)\n",
    "from sklearn.ensemble import BaggingRegressor\n",
    "# 2. Create an instance of the estimator\n",
    "knn_for_bagging = KNeighborsRegressor(n_neighbors=20, weights='distance', metric='euclidean')\n",
    "\n",
    "bagging = BaggingRegressor(base_estimator=knn_for_bagging, n_estimators=15, max_features=0.75, \n",
    "                            random_state=55, n_jobs=-1)\n",
    "# 3. Use the trainning data to train the estimator\n",
    "bagging.fit(X_train, y_train)\n",
    "# 4. Evaluate the model\n",
    "models.loc['train_mse','Bagging'] = mean_squared_error(y_pred=bagging.predict(X_train), \n",
    "                                                    y_true=y_train)\n",
    "\n",
    "models.loc['test_mse','Bagging'] = mean_squared_error(y_pred=bagging.predict(X_test), \n",
    "                                                   y_true=y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C. Random Forests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 1. Import the estimator object (model)\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "# 2. Create an instance of the estimator\n",
    "RF = RandomForestRegressor(n_estimators=50, max_depth=16, random_state=55, n_jobs=-1)\n",
    "# 3. Use the trainning data to train the estimator\n",
    "RF.fit(X_train, y_train)\n",
    "# 4. Evaluate the model\n",
    "models.loc['train_mse','RandomForest'] = mean_squared_error(y_pred=RF.predict(X_train), \n",
    "                                                    y_true=y_train)\n",
    "\n",
    "models.loc['test_mse','RandomForest'] = mean_squared_error(y_pred=RF.predict(X_test), \n",
    "                                                   y_true=y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## D. Boosting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 1. Import the estimator object (model)\n",
    "from sklearn.ensemble import AdaBoostRegressor\n",
    "# 2. Create an instance of the estimator\n",
    "boosting = AdaBoostRegressor(n_estimators=50, learning_rate=0.05, random_state=55)                             \n",
    "# 3. Use the trainning data to train the estimator\n",
    "boosting.fit(X_train, y_train)\n",
    "# 4. Evaluate the model\n",
    "models.loc['train_mse','Boosting'] = mean_squared_error(y_pred=boosting.predict(X_train), \n",
    "                                                    y_true=y_train)\n",
    "\n",
    "models.loc['test_mse','Boosting'] = mean_squared_error(y_pred=boosting.predict(X_test), \n",
    "                                                   y_true=y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>KNN</th>\n",
       "      <th>Bagging</th>\n",
       "      <th>RandomForest</th>\n",
       "      <th>Boosting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>train_mse</th>\n",
       "      <td>78.503</td>\n",
       "      <td>112862</td>\n",
       "      <td>142186</td>\n",
       "      <td>1.82036e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_mse</th>\n",
       "      <td>744451</td>\n",
       "      <td>688060</td>\n",
       "      <td>376764</td>\n",
       "      <td>1.81305e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              KNN Bagging RandomForest     Boosting\n",
       "train_mse  78.503  112862       142186  1.82036e+06\n",
       "test_mse   744451  688060       376764  1.81305e+06"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAE/CAYAAADmGaF6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHOlJREFUeJzt3Xu8Z1Vd//HX2xkElLsQDaCM9NNIRBHHSkVELUXwboVK\ngVoZZXn3F6b1o8cvCy+oP4oiLFQKBc28EJZiKWCiMIMDAwqKXFLkIrcB5CIMn98fex35znHOOTPD\nzPqeOef1fDy+D/ZZe++119r7y+z3d+31PSdVhSRJUi8PGncDJEnS/GL4kCRJXRk+JElSV4YPSZLU\nleFDkiR1ZfiQJEldGT4kPWBJDkjy3SS3Jzlw3O0ZtyS/kuSCcbdjbSU5IskX13LbU5K8Y2O3SXOb\n4UPaCNpNeOJ1X5I7R34+9AHU+7UkvznN+j2TVJJzJpXvkuTeJJeMlB3Q6luZ5KYkX0myT1t3RNv+\n9kmvh01x6HcC766qrarqP9a3fyNtOyXJ3e2YNyX5jyT/64HW20tVfbGqHr+h612X6yvNZoYPaSNo\nN+Gtqmor4H+A54+UnbyRD78K2CnJo0bKfhP47sQPLUR8BngPsD2wG0OA+PHIPl8e7Ud73TjFMXcH\nLl6fxiZZOMWq/9vO327AzcDfr0/9D+D4s9WM11ea7Qwf0hgkWZDkT5NcnuSGJCcn2a6te2j75H9T\nkluSfD3J9kmOAZ4E/EMbEThmmkP8M3DYyM+/BZw08vMvAHdW1Ser6r6quqOq/r2qvrkeffk+sAvw\nhSS3t7JHJPlc68O3kxw+sv3RST6a5NQktwEvm67+qroD+ASwz6Tj/l6SS9sxTk+y68i6g5N8p52/\nD4yOGLVRnf9KclySm4Ejp6uvXavjkvywjRJdkOTn27oXJrkkyW1Jvpfkda38wCSXjbRn7yRnt/Zc\nmOS5I+tOaW38fKvnv5PsPsNpn+n6znTMn2nX59Y2irL7pH0f287RzUm+leRFa2pEkp9to1K3JLkx\nyX/N0G4JMHxI4/IW4NnAfgyf7O8B3t/W/Q6wENgV2BH4Q+DHVfVm4Dzgd9ooxJunqf8k4NAM9gXu\nA0bnIHwL2DLJPyZ5zkTwWR9VtRtwPfDsNlIBQ1i4FFgEvAJ4f5Knjuz2UuAjwLbAJ6erP8nWDAFl\n9GZ+CPAG4PnAzsA3GG7IJFkEnAq8EdgJ+AHwxEnV7g8sZzi/x0xXH/A8YF/g5xhGiV7BMBIDcCJw\nWFVtzRCOzl5D+7cATgc+3drzVuATSR45stkrgLcBOwDXAH8+3Tlhhuu7Fsc8Abip9fX3gVeP7LsN\ncAbwj+38HAacOMVjrz9muM47Mlzro2ZotwQYPqRxOQI4sqp+UFV3MdxsDkkShiCyE/BzVXVvVZ1X\nVT9al8qr6nKGm+7+DDePkyatvxF4KrAZ8CHgh0n+NcmOI5s9vX2inXit1WOV9jjg8cCfVNXdVbWU\nIWj81shmZ1bV59qoy51TVPX2JLcAtzLc/F81su4I4C+q6ttVdQ/D+dsvyc4MAeK8qvq3tu693B8W\nJlxeVR+sqlXt+NPVdw+wDbBnO3cXV9X1rZ5VwF5Jtq6qG6vqG2vox9OAAt5XVfdU1ecZbu6HjGzz\n8ao6vx37o0wa5Zlspus73TFbMHkB8I6qurOqlgOjjwJfDFxUVSe383MecBpDYJzsHoZRr0dU1Y+r\n6qzp2i1NMHxInbWA8XDgcxM3doZP2g8CHsbwifNM4F+SfD/JXyZZsB6HOonhE+2vs/rNBYCquqiq\nDquqXRhudj/HMAdkwplVtd3Ia6+1PO4uwA8nhYqrGEZyJnxvLep5Z1VtB+zBcJMf/eS9O3D8yPn7\nIXAvwyjSLqP1V9V9wNWT6p58/Onq+3eGa/L3wLVJ/jbJxAjPCxluyv/THlM8aQ392AX4n1r9r3hO\nPh/XjizfAWzFzKa7vtMd82eBsPo5uGpkeXdg/9HgydDHRWtowzsZQtCXklyW5E1r0W7J8CH11m4I\nVwPPnHRz36KqbmijBX9WVXsyfLL9de6fF7Euf4b6423fC6vq2uk2rKqLgX8CHrvOHfppP2CYELnl\nSNkjWD0ArHU/quoKhsdUf53kwa34e8ArJ52/LatqGcNji90m9k/yIFa/0a/p+FPWV4P3VdUTgMcx\njOq8vrXtnKp6HsPjiy8wjFqs6Xw8YlLZ5POxPqa7vtMd81qG/j980roJ3wO+MOlcbFVVb5jcgKpa\nWVWvr6rdGQLKOyY9XpPWyPAhjcfxwNFJHg4/mQD4/Lb8K0ke026atzJ8Ar+v7Xcdw0jAjKrqFuAA\nhmf6q2mTEd8wMqlyMcNjgK89gD5NuAxYAfxFks3bnITDuX8OxTqrqtMYzsXEo5fjGW50ExM/t08y\n8Vjgs8AvJTkowzdZ3sQwV2M6U9aX5JeTLGl1/YjhG0H3ZZgY/LI2R+Ie4Dbuv06jzgYe1M73wiS/\nyjDf5+Prcy4mTHd9pztme8x3GvDnSbZM8jhg9OvfnwaekOSQJJsleXA7B4+efJAkL0iyRxvNW8kw\nQrWmcyCtxvAhjce7gS8C/5XhGx9fZZjXAMOn9M8w3MwuAj7HMIEShkmph7VvIbx7poNU1blVdeUa\nVt3KMNn1vCQ/Ar4CnEv75kdzQH7693zsvRbHLIZP5I9h+JR9KvDWqvrKTPvO4L3AkUk2q6qPAX8D\n/GuSWxkmj/5qO/41wMuBY4EbGEZBVgB3T9PmKesDtgM+DNwCXM7wiOL/tXWvbj+vZJh7MfoNlIm6\n72KYtPprwI3A+4BD2ryNB2Sq67sWx/w9htGa6xgeJ31oZN+bgecwBL1rGEZR/oJhftBkvwB8ieG9\nehbw3qo6Zw3bSavJ6o8EJWluaSMW1zL8rhVvjNIs4MiHpDknyXOTbNu+2fF/GCZxLhtzsyQ1hg9J\nc9H+wBUMv3/kWcCLq+rH0+8iqRcfu0iSpK4c+ZAkSV0ZPiRJUleb2l9z3KTsuOOOtXjx4nE3Q5Kk\nLpYtW3ZDVe0003aGj41o8eLFLF26dNzNkCSpiyRXzbyVj10kSVJnhg9JktSV4UOSJHVl+JAkSV0Z\nPiRJUleGD0mS1JXhQ5IkdWX4kCRJXRk+JElSV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV\n4UOSJHW1cNwNmMtWXL2SxUeePu5mSJL0U648+uCxHduRD0mS1JXhQ5IkdWX4kCRJXRk+JElSV4YP\nSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJUlezMnwkWZVkeZILkpyf\n5CkbuP4/mfTzVzdk/ZIkaWqzMnwAd1bVPlX1eOBtwF9t4PpXCx9VtUHDjSRJmtpsDR+jtgFuBsjg\nPUkuSrIiySEzlC9KclYbRbkoydOSHA1s2cpObtvd3v57QJIvJ/mXJJckOTlJ2rqDWtmyJMcm+bdx\nnAxJkjZ1C8fdgClsmWQ5sAWwCHhmK38JsA/weGBH4LwkZwFPmaL8FcDnq+qdSRYAD6mqs5P8YVXt\nM8WxnwDsBfwA+G/gqUmWAn8P7F9VVyT52EbosyRJ88JsHfmYeOyyJ3AgcFIbgdgP+FhVraqq64Az\ngSdNU34e8KokRwF7V9Vta3Hsc6vq+1V1H7AcWAzsCVxeVVe0baYMH0lek2RpkqWr7li5Hl2XJGlu\nm63h4yeq6hyG0Yyd1mPfs4D9gauBDyc5bC12u3tkeRXrODpUVSdU1ZKqWrLgIduuy66SJM0Lsz58\nJNkTWADcCJwNHJJkQZKdGILFuVOVJ9kduK6qPgj8A7Bvq/aeJJutQzMuBfZIsrj9fMgD7JYkSfPW\nbJ/zARDg8KpaleRTwJOBC4AC/ndVXTtN+eHAW5PcA9wOTIx8nABcmOT8qjp0psZU1Z1J/gD4jyQ/\nYnicI0mS1kOqatxt2CQk2aqqbm9zT44DvlNV759un80XPaoWHf6BPg2UJGkdXHn0wRu8ziTLqmrJ\nTNvN+scus8jvttGYi4FtGb79IkmS1tFsfewy67RRjmlHOiRJ0swc+ZAkSV0ZPiRJUleGD0mS1JXh\nQ5IkdWX4kCRJXRk+JElSV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJX/nr1jWjvXbdl6Ub4wz2SJG3K\nHPmQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJUleGD0mS1JXhQ5IkdWX4kCRJXRk+JElS\nV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJUleGD0mS1JXhQ5Ik\ndWX4kCRJXRk+JElSV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJ\nUleGD0mS1JXhQ5IkdWX4kCRJXS0cdwPmshVXr2TxkaePuxnSOrny6IPH3QRJc5wjH5IkqSvDhyRJ\n6srwIUmSujJ8SJKkrgwfkiSpK8OHJEnqyvAhSZK6MnxIkqSuDB+SJKkrw4ckSerK8CFJkroyfEiS\npK7mVfhIcvvI8kFJvp1k9yRHJbkjyc9MsW0lOWbk57ckOapbwyVJmkPmVfiYkORZwLHAc6vqqlZ8\nA/DmKXa5G3hJkh17tE+SpLls3oWPJPsDHwSeV1XfHVl1InBIkh3WsNu9wAnAGzs0UZKkOW2+hY/N\ngU8DL6qqSyatu50hgLx+in2PAw5Nsu1GbJ8kSXPefAsf9wBfBX57ivXHAocn2Xryiqq6FTgJeN10\nB0jymiRLkyxddcfKB9peSZLmnPkWPu4DfgP4xSR/MnllVd0CfBR47RT7f4AhuDx0qgNU1QlVtaSq\nlix4iIMkkiRNNt/CB1V1B3AwwyOUNY2AvA/4PWDhGva9Cfg4U4+cSJKkGcy78AE/CREHAu9I8oJJ\n624APsUwP2RNjgH81oskSevppz7dz2VVtdXI8veAR7YfPztpuzcBb5piv+uAh2zclkqSNHfNy5EP\nSZI0PoYPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJUleGD0mS1JXh\nQ5IkdWX4kCRJXc2rPyzX2967bsvSow8edzMkSZpVHPmQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAk\nSV0ZPiRJUleGD0mS1JXhQ5IkdWX4kCRJXRk+JElSV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9J\nktSV4UOSJHVl+JAkSV0ZPiRJUleGD0mS1JXhQ5IkdWX4kCRJXRk+JElSV4YPSZLUleFDkiR1ZfiQ\nJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJUleGD0mS1JXhQ5IkdWX4kCRJXS0cdwPmshVX\nr2TxkaePuxmax648+uBxN0GSfoojH5IkqSvDhyRJ6srwIUmSujJ8SJKkrgwfkiSpK8OHJEnqyvAh\nSZK6MnxIkqSuDB+SJKkrw4ckSerK8CFJkroyfEiSpK5mdfhIsirJ8iQXJDk/yVM2wjGWJDl2Q9cr\nSZLWbLb/Vds7q2ofgCTPAf4KePqGPEBVLQWWbsg6JUnS1Gb1yMck2wA3AyTZKsl/ttGQFUleOLFR\nkj9NcmmSryT5WJK3tPInJbmwjaS8J8lFrfyAJP/Wlo9KcmKSLye5PMnrZqpXkiStm9k+8rFlkuXA\nFsAi4Jmt/C7gxVV1a5Idga8l+SywBHgp8HhgM+B8YFnb50PA71bVOUmOnuaYewLPALYGLk3yd8A+\n09QrSZLWwWwPH6OPXZ4MnJTksUCAv0yyP3AfsCuwM/BU4DNVdRdwV5LT2r7bAVtX1Tmt3o8Cz5vi\nmKdX1d3A3Umun67eNUnyGuA1AAu22ekBdF2SpLlptoePn2gjFjsCOwEHtf8+saruSXIlw+jIhnD3\nyPIq1vEcVdUJwAkAmy96VG2gNkmSNGdsMnM+kuwJLABuBLYFrm/B4xnA7m2z/waen2SLJFvRRjeq\n6hbgtiS/1LZ72Toefo31SpKkdTfbRz4m5nzA8Kjl8KpaleRk4LQkKxi+qXIJQFWd1+Z+XAhcB6wA\nVrb9fxv4YJL7gDNHymc0Q72SJGkdzOrwUVULpii/AXjyFLu9t6qOSvIQ4Czunxh6cVU9DiDJkbSv\n11bVl4Evt+WjJh3nsWtRryRJWgezOnyspxOSPIZhDshHqur8Vn5wkrcx9Pkq4JUbqF5JkrQO5lz4\nqKpXTFF+KnDqhq5XkiStm01mwqkkSZobDB+SJKkrw4ckSerK8CFJkroyfEiSpK4MH5IkqSvDhyRJ\n6srwIUmSujJ8SJKkrgwfkiSpqzn369Vnk7133ZalRx887mZIkjSrOPIhSZK6MnxIkqSuDB+SJKkr\nw4ckSerK8CFJkroyfEiSpK4MH5IkqSvDhyRJ6srwIUmSujJ8SJKkrgwfkiSpK8OHJEnqyvAhSZK6\nMnxIkqSuDB+SJKkrw4ckSerK8CFJkroyfEiSpK4MH5IkqSvDhyRJ6srwIUmSujJ8SJKkrgwfkiSp\nK8OHJEnqyvAhSZK6MnxIkqSuDB+SJKkrw4ckSerK8CFJkroyfEiSpK4MH5IkqSvDhyRJ6srwIUmS\nulo47gbMZSuuXsniI08fdzPmrCuPPnjcTZAkrQdHPiRJUleGD0mS1JXhQ5IkdWX4kCRJXRk+JElS\nV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV3NGD6SrEqyPMlFSU5Lst2G\nOHCSxUku2kB1fTjJFa2dy5O8bkPUO8WxDkjylI1VvyRJc93ajHzcWVX7VNVjgZuA127kNq2vt7Z2\n7lNVx67tTkkWrONxDgAMH5Ikrad1fexyDrArQJKtkvxnkvOTrEjywla+OMm3knwwycVJvpBky7bu\niUkuSHIBIyEmyRZJPtTq+UaSZ7TyVyb5dJIzklyZ5A+TvKlt87UkO0zX2CQvb3VelORdI+W3Jzmm\ntePJrV1nJlmW5PNJFrXtXpfkm0kuTHJKksXAEcAb2wjL09bx/EmSNO+tdfhoIwTPAj7biu4CXlxV\n+wLPAI5JkrbuUcBxVbUXcAvw0lb+IeCPqurxk6p/LVBVtTfwcuAjSbZo6x4LvAR4EvBO4I6qegJD\nEDpspI73jDx22TvJLsC7gGcC+wBPSvKitu1Dga+3dnwd+Gvg16rqicCJ7TgARwJPqKrHAUdU1ZXA\n8cD72wjL2Wt7/iRJ0mBtwseWSZYD1wI7A2e08gB/meRC4IsMIyI7t3VXVNXytrwMWNzmimxXVWe1\n8n8aOcZ+wD8DVNUlwFXAo9u6L1XVbVX1Q2AlcForXwEsHqlj9LHLCoaw8uWq+mFV3QucDOzftl0F\nfLIt/zxDwDmj9fMdwG5t3YXAyUl+E7h3Lc4VSV6TZGmSpavuWLk2u0iSNK+s9ZwPYHeGwDHxuORQ\nYCfgiW39dcDEaMXdI/uvAhY+gDaO1nXfyM/3PYB676qqVW05wMUjwWXvqnp2W3cwcBywL3BekhmP\nV1UnVNWSqlqy4CHbrmfzJEmau9b6sUtV3QG8DnhzuwlvC1xfVfe0ORq7z7D/LcAtSfZrRYeOrD57\n4uckjwYeAVy61r1Ys3OBpyfZsT0yejlw5hq2uxTYKcmT2/E3S7JXkgcBD6+qLwF/zNDfrYDbgK0f\nYNskSZq31mnCaVV9g+FRxMsZHmMsSbKCYe7FJWtRxauA49rjjYyU/y3woFbXqcArq+ruNVWwDm29\nhmHOxpeAC4BlVfWZNWz3Y+DXgHe1CajLGb7NsgD459ambwDHtgB1GvBiJ5xKkrR+UlXjbsOctfmi\nR9Wiwz8w7mbMWVceffC4myBJGpFkWVUtmWk7f8OpJEnqyvAhSZK6MnxIkqSuDB+SJKkrw4ckSerK\n8CFJkroyfEiSpK4MH5IkqSvDhyRJ6srwIUmSujJ8SJKkrgwfkiSpq4XjbsBctveu27LUP34mSdJq\nHPmQJEldGT4kSVJXhg9JktSV4UOSJHVl+JAkSV0ZPiRJUleGD0mS1JXhQ5IkdWX4kCRJXRk+JElS\nV4YPSZLUleFDkiR1ZfiQJEldGT4kSVJXhg9JktSV4UOSJHWVqhp3G+asJLcBl467HR3sCNww7kZ0\nYl/nnvnST5g/fZ0v/YTZ19fdq2qnmTZa2KMl89ilVbVk3I3Y2JIsnQ/9BPs6F82XfsL86et86Sds\nun31sYskSerK8CFJkroyfGxcJ4y7AZ3Ml36CfZ2L5ks/Yf70db70EzbRvjrhVJIkdeXIhyRJ6srw\nsREkOTDJpUkuS3LkuNszlSQPT/KlJN9McnGS17fyo5JcnWR5ex00ss/bWr8uTfKckfInJlnR1h2b\nJK188ySntvKvJ1k8ss/hSb7TXod36O+VrY3LkyxtZTskOaO14Ywk22/KfU3y8yPXbXmSW5O8Ya5c\n0yQnJrk+yUUjZWO9hkke2ba9rO374I3Y1/ckuSTJhUk+lWS7Vr44yZ0j1/f4TaWvU/RzrO/Xztf0\n1JF+XplkeSvfZK/pWqkqXxvwBSwAvgvsATwYuAB4zLjbNUVbFwH7tuWtgW8DjwGOAt6yhu0f0/qz\nOfDI1s8Fbd25wC8DAf4deG4r/wPg+Lb8MuDUtrwDcHn77/ZtefuN3N8rgR0nlb0bOLItHwm8ay70\ndeS9eC2w+1y5psD+wL7ARbPlGgIfB17Wlo8Hfn8j9vXZwMK2/K6Rvi4e3W5SPbO6r1P0c6zv157X\ndNL6Y4A/29Sv6dq8HPnY8H4RuKyqLq+qHwOnAC8cc5vWqKquqarz2/JtwLeAXafZ5YXAKVV1d1Vd\nAVwG/GKSRcA2VfW1Gt7BJwEvGtnnI235X4BntZT+HOCMqrqpqm4GzgAO3MBdXBuj7fsIq7d7U+/r\ns4DvVtVV02yzSfWzqs4CblpDH8ZyDdu6Z7ZtJx9/g/e1qr5QVfe2H78G7DZdHZtCX6e4plOZc9d0\nQjvubwAfm66OTaWvMzF8bHi7At8b+fn7TH9DnxXa8NwTgK+3oj9qQ7sn5v5h7Kn6tmtbnly+2j7t\nH82VwMOmqWtjKuCLSZYleU0r27mqrmnL1wI7T273pPZtKn2F4ZPP6D9kc/Gawniv4cOAW0YCQc//\n31/N8Kl3wiPb8PyZSZ7Wyjblvo7r/Tqua/o04Lqq+s5I2Vy7pj9h+BBJtgI+Cbyhqm4F/o7hsdE+\nwDUMQ4FzwX5VtQ/wXOC1SfYfXdk+RcyJr3+157YvAD7RiubqNV3NXLqG00nyduBe4ORWdA3wiPb+\nfhPw0STbjKt9G8C8eL9O8nJW/7Aw167pagwfG97VwMNHft6tlc1KSTZjCB4nV9W/AlTVdVW1qqru\nAz7I8CgJpu7b1aw+/Dva55/sk2QhsC1w4zR1bTRVdXX77/XApxj6dV0bxpwYzrx+crsntW+T6CtD\nwDq/qq6DuXtNm3FewxuB7dq2k+vaKJK8EngecGgLW7THEDe25WUMcyEezSba1zG/X8dxTRcCLwFO\nnSiba9f0p/SYWDKfXgx/L+dyhslQExNO9xp3u6ZoaxieF35gUvmikeU3MjxjBdiL1Sd7Xc7Uk70O\nauWvZfUJUB9vyzsAVzBMftq+Le+wEfv6UGDrkeWvMsxHeA+rT1Z896be13bMU4BXzcVryqSJeOO+\nhgyjS6MT9v5gI/b1QOCbwE6TtttppG97MNxAdthU+rqGfo71/drzmo5c1zPn0jWd8Tz0OMh8ewEH\nMXxz5LvA28fdnmnauR/DEPWFwPL2Ogj4J2BFK//spH8I3t76dSlthnUrXwJc1Nb9Dff/Arst2pv7\nsvY/zB4j+7y6lV/GyI1yI/V1j/aP1gXAxRPXheGZ538C3wG+yMjNchPu60MZPtFsO1I2J64pw7D0\nNcA9DM+nf3vc17C9t85t5Z8ANt+Ifb2M4dn9xP+vEzeal7b39XLgfOD5m0pfp+jnWN+vPa9pK/8w\ncMSkbTfZa7o2L3/DqSRJ6so5H5IkqSvDhyRJ6srwIUmSujJ8SJKkrgwfkiSpK8OHJEnqyvAhSZK6\nMnxIkqSu/j9lW61YgwtUnwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1e433cd2a58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "models.loc['test_mse'].sort_values().plot(kind='barh', ax=ax)\n",
    "ax.set_title('Test MSE for Regression Models');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAFNCAYAAACQfxyRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXucXVV1+L8rEyRpBqJGkiKQTBwTFIOgPDL1pwZfEUmU\noBYwZHxRqdX6iFoZWqNoagm2iq/6BpGkQaiaSDNBYlWirc0gWpX4CowzCSBOMGpgYkLJZP3+OGff\n2Xffc8499zX3zsz6fj7zmXvPY599zp25a+31FFXFMAzDMAxjSrMnYBiGYRhGa2BKgWEYhmEYgCkF\nhmEYhmHEmFJgGIZhGAZgSoFhGIZhGDGmFBiGYRiGAZhSYASIyDkicl+z5zFZEZHXish/ee+HReRJ\nVYxziYhsq+/sWgsRuVJENjR7HpUQfr51HLcpz0JErhKRt4/1dZMQkUEReWGDxv6qiLykEWO3GqYU\njAPiP/aDsYD4rYhcLyLtzZ5XrYiIisiB+L6GReSPY3z9lleAVLVdVX+ddYyIdMTPcqp33r+p6tLG\nz9CYrIjIccCrgc/G7+v2/yQit4vIX9VjrJTxVUSeXMEpVwP/2Kj5tBKmFIwfXqqq7cDpwDOAK5o8\nn3pxWiz42lX1sZWe7AvCVqOV52bkwz7DTF4LbFXVg82eSKNR1TuAY0XkzGbPpdGYUjDOUNXfArcR\nKQcAiMgyEflfEXlIRO4VkSu9fW4V+RoR2SMivxORf/D2T48tD38QkZ8DZ/nXE5Gnxlr7H0XkZyLy\nMm/f9SLyKRG5NV7p/7eI/LmIfDQe75ci8oxq7lNE3iAi94jI70XkFhF5ordPReTNInI3cHe87Ski\n8s34+F+JyIXe8eeJyM9F5GERuV9E3iUiM4BbgSd6loonlkykdF7Xi8hn4ms9LCLbRWReDXObFd/f\nQyJyB9AZXK+wook/qw+LyG4R2S8i/yUi04Hvxof/Mb6Pv0hwQzxLRH4Qn/cDEXmWt+92EVkbf34P\ni8g2EXlCvG+aiGwQkX3x38APRGROwnO5XES+Emz7mIh8PH79WhH5dTz+gIhcUu5Zx+c9MX4+v4//\nHt4QHDJNRG6Kx/2RiJwWzOn+eN+vROQF8fYpItIjIv3xfd0sIo+P97n/l0tFZA/w7fjv+2+Def1E\nRF4ev6768w3GLHedj0n0//2QiPxQRJ6TMk7Jil0803qZ+8/1ece8BNgen5f4/1TNtUTkg8BzgE/G\n43wy5T674/+FfeJ9p8X7zhaR/4nHfUBEPikij4n3uf+Xn8TjXyQijxORLSLyoETfXVtE5MTgkrcD\ny1KexcRBVe2nxX+AQeCF8esTgbuAj3n7zwFOJVLyng4MASvifR2AAp8HpgOnAY8AT433rwO+Bzwe\nOAnYCdwX7zsKuAf4e+AxwPOBh4GT4/3XA78DzgCmAd8GBohMim1E5rbvZNyXAk9O2P78eNxnAkcD\nnwC+G5z3zXjO04EZwL3A64CpRJaU3wGnxMc/ADwnfv044Jnec7svuPazgT9mzPn6+Bk8N57bx4D/\nqmFuXwZujo9bBNyfMN6T49f/SvTFdEL8fJ8Vz8F9xlO9817rxonn8gegO57Dq+L3s+L9twP9wMJ4\nzrcD6+J9fw38B/Bn8TXPAI5NeC7zgD8Bx8Tv2+Ln3hXf20Pe383xwNNy/u1/F/gU0d/X6cCDwPPj\nfVcCjwKvJPpbfRfR399RwMnxc3+i93/QGb9+G7CD6H/paCLz943B/8sN8bynE/09/7c3p1OAP8bn\n1vT5Bveaep34/SpgVnyddwK/BaZ5z2JDxt/1IKPfIVn3n+vzjo99EDgr+B4Kr1vVtYj+Bv8q4+/i\nFGCY0f/DjwCHvXs8g+hvb2r8mf4CeHvad0/8XF8Rz+UY4N+BzcE13wF8rVHf863y0/QJ2E+ODyn6\nhx4mEkYKfAt4bMbxHwWuiV93xOec6O2/A7g4fv1r4Fxv32WMKgXPib94pnj7bwSujF9fD3ze2/cW\n4Bfe+1PJFrBKJCz+GP98PN5+LfAh77h2oi//Du+853v7LwK+F4z9WeB98es98RfQscExJV9iOT6L\n64EvB3MbAU6qdG7xF+GjwFO8ff9EglJApPAdJHK3hHNyn3GaUtAN3BGc8z/Aa+PXtwPv8fa9CfhG\n/Pr1wPeBp+d4Nv8FvDp+/SKgP349I/58XwFMr+BZnxQ/22O8bVcB18evrwR2ePumECuA8TPbC7wQ\nOCoY9xfAC7z3x8efgxMgCjzJ238McACYF7//IHBdPT7f4LzU66Qc/wf390BlSkHW/VfyeYf3lnTd\nqq5FeaXgvRT/H84A/s/dY8Lxbwc2hf9XGeOfDvwh2PYG4Nt5/37H64+5D8YPK1T1GKJ/vKcAT3A7\nRGSxiHwnNn3tB97o74/5rff6T0TCDOCJRCsdx27v9ROBe1X1SLD/BO/9kPf6YML7cgGRz1TVx8Y/\nb/WuW5iHqg4D+4Lr+nOeByyOTYV/lChg8RLgz+P9rwDOA3ZLZO7/izJzKkfh2vHcfh/PudK5HUf0\n5Zj2/H2eQLRa7q9ivkXP07uO/zzT/j7WE7mrviwivxGRD4nIUSnX2UhkhQBYGb9HVQ8QCc83Ag+I\nSK+IPCXnvH+vqg9nzNv/LI4A9xFZB+4hEgRXAntF5Msy6h6aB2zyPo9fECkfc1LGfRjoBS6ON70K\n+DdvrHp8vuWug0Rur19I5AL6IzCT0v/zPGTdfyWf9x+IFJmxuFZI0fdW/De2z70XkYWxC+C3IvIQ\nkTKW+qxE5M9E5LOxO+IhIgvVY0WkzTvsGCLldkJjSsE4Q1W3E61W/8XbvBG4hWi1OhP4DCA5h3yA\naEXmmOu9/g1wkohMCfbfX+G0K+U3RF8mQMFfOSu4rnqv7wW2e8rFYzUKXPwbAFX9gaqeD8wGNhOZ\nc8MxKqHwvCTKAnl8POdK5/Ygkckz7fn7/A44RLJPutx9FD1P7zplP0dVfVRV36+qpxC5K5YTmbmT\n+HfgnNgXewGxUhCPc5uqvohopfhLIndWOX4DPF5EfMETztv/LKYQmal/E19zo6o+m+jelSiCHKLP\n5CXBZzJNVdP+viCykL0qViinAd/xxqrH55t5nTh+4N3AhcDjNArK3U/y//kBIjO4ey5tRAqKI/X+\nK/y8f0rkcnIk/R1We61yf9NF31si8mdE3xGOTxP9nS1Q1WOJXKBZ34nvJHI5LY6Pf64b2jvmqcBP\nysxr3GNKwfjko8CLZDSo6hiiFdUhETmbaJWWl5uBK+JAmxOJXACOPqJV47tF5CgROQd4KZGftJHc\nCLxORE4XkaOJtPw+VR1MOX4LsDAOPDoq/jlLoiDJx0iUsz9TVR8lclc4y8cQMEtEZlY4v/NE5Nlx\n4NJaIhP2vSnHps5NVUeArwFXxiuVU4DXJA0Sr4KvAz4SB3C1SRRQeDSR8DkCpNUz2BrPYaWITBWR\ni4h8slvK3aiIPE9ETo0Fy0NEpt8jSceq6oNEZt8vAgOq+ot4jDkicn6s3D1C5ApLHCMY714i8/JV\nEgWlPR24FPDz8c8QkZdLlCXw9nj8HSJysog8P34+h4isVu6anwE+KHGAqIgcJyLnl5nOViLl4gPA\nTZ71rC6fb47rHEOkYDwITBWR9wLHpoyxiygAc1m88n4Pkd/dkXr/lXze8VyXeO+T/p+qvdYQ6X/P\nAF8Blnv/hx+gWJ4dE485HFul/iY4Pxz/GKK/kT9KFAj5voRrLiEKppzQmFIwDom/fG8g8qtB5AP+\ngIg8HG+7Oe3cBN5PZNIcALYRmfTcdf6PSAl4CdFK9VNEPuNf1noPWajqfwJrgK8SrQg6GTWpJh3/\nMLA0PuY3RKbwqxn9IuwGBmOz4BuJzLvE93Ej8OvYvPlEEXmOiAyXmeJGoi+N3xMFNK2qYW5/S2Sq\n/y2RBeiLGdd9F1GQ6Q/ia19NFO/xJyL/83/H99EVzGEf0SrsnUQm1ncDy1X1d2XuEyIz+FeIvmB/\nQRRtvj7j+I1EfvyN3rYpREFav4nnvYT4SzrH834VkZ//N8AmojiR//T2f53INeECKV8eK39HEwXR\n/o7o2c5mNI33Y0SWtW3x/8wOYHHGHFDVR4gEfNG91fnzTb0OkZn9G0QCfzeRopOoiKrqfqLvhC8Q\nWVUOELlVHFn3X8nnfQORgjw9vm7J/1MN1/oY8EqJMgE+nnCPPwPeHD+jB4g+f/8e30W0OHqYyCp1\nUzDElcCX4nleSLTQmk7097KD6FkXEJGzgGGNUhMnNKJarQXVMCYfInI9UTDVe5o9F8NoNiLyT8Be\nVf1os+fSSETkq8C1qrq12XNpNFaYwzAMw6gKVf37Zs9hLFDVVzR7DmOFuQ8MwzAMwwDMfWAYhmEY\nRoxZCgzDMAzDAEwpMAzDMAwjZtIFGj7hCU/Qjo6OZk/DMAzDMMaEH/7wh79T1ePKHzkJlYKOjg7u\nvPPOZk/DMAzDMMYEEUktrx1i7gPDMAzDMABTCgzDMAzDiDGlwDAMwzAMwJQCwzAMwzBiTCkwDMMw\nDAMwpcAwDMMwjBhTCgzDMAzDAEwpMAzDMAwjxpQCwzAMwzAAUwoMwzCMJrBm8046r9jKms07G35e\ntddqCpddBp/4RNMub0qBYRjGJGUshWV4rY19exhRZWPfnorOX79jNyOqrN+xO3XetVyraQrE9deD\nCHz+8/DWt47ttT1MKTAMw2gizVzFOmGZJWDrfS0nmFcunkubCCsXz63o/HBbva9VToFoyOclAq97\n3ej7l7ykfmNXiCkFhmEYTaTSFXM98YVko68fCua1KxbRf9V5rF2xqHBMmsBds3lniULgxqz2Wnnn\nGc6rrp+XSPTjowpbt9Y+dpWYUmAYxqSkVfzMla6YQ2q5j7UrFtHdNa+m61dyrXKCOU3ghu/bROju\nmsfGvj2J912JEpDnXH9e4edV1fO/6aZSZeDmmyOFoMmItsAkxpIzzzxTrXWyYRidV2xlRJU2Efqv\nOq/Z06maVrkPt4runD2D/r0HWLl4bsVC2Y2xcvFc+gb2sWtomJnTp7L/4GEEWDCnnbuHhgmlljvm\nqDbhyBEqurZ/zbRzso5xzx8oKAuZ1w6VAWi4MiAiP1TVM/Mca5YCwzAmJbWu0FuFVrkPt5reNTRc\ntXndX6XvGhoGYP/BwwBMEWHb6iVMCYSqeMc8OqIl1w5X8nmCEMNjsiwP7vkLZN93mqugxRbmZikw\nDMMwqsJfQQMFS4FbzXd3zavKhA+w9JrtBcUAbyx3zfZpbew/eJiFc9qLjvMVpI19eziiisbb+686\nr8iysnLxXNbv2I0Aq7y5VmN9SbUmbN8O55xTfPBjHgOPPFLNY6kKsxQYhmEYDcdfZbvVtL+aryUY\nb9vqJUXvnaB11xk+NAJA/94DdHfNw63BnVB2c3MKgVMUfMuKm98UkSJBnmV9SYshcPMCRveLlCoE\nqmOqEFSKKQWGYRhGVaQJz3q5NFwQZHfXvMxrr12xqEQRcfu7u+YVmf59V8DKxXMLZv+l12wvCPMs\nd0G57IONfXvoX7eMtRecWrzjT39qOVdBEuY+MAzDMBLJE4TXKuQNGAzdBX6gIFDkMkgaM/M6v/0t\nHH986YWbLGfNfWAYhmHUTDNrKFRKnroH7j6UUovCwjntJdaNpPtPdBNA5CoIFYIWDCQshykFhmEY\nRiLlCvm0On5JZBgtdiTeaz8WIlQqstwgTmFYe8GppVkFO3eOO2XAYe4DwzAMIzdh9H6SKb3ebodq\nx+vo6S289uMP6jG/9371J3zglaeX7mhBmWruA8MwDKMhhNH7aRUI6+V2cHEAbsXfecXWQlCgHxyY\nxMI57YXX/nxqnp9IqUIwDl0FSTRMKRCR60Rkr4js9LbdJCI/jn8GReTH8fYOETno7fuMd84ZInKX\niNwjIh8Xiew0InJ0PN49ItInIh2NuhfDMAwjIozedwqC71aoV/aBUwgcLlPAFUgKCyX5c1izeWeh\nfsHM6VNT0xLD8zJJKkD0qU9NCGXA0TD3gYg8FxgGblDVEvuMiHwY2K+qH4gF+paU4+4A3gr0AVuB\nj6vqrSLyJuDpqvpGEbkYuEBVLyo3L3MfGIZRKa0Uhd9Kc/HJU/AnnHu5e/EzA1zGgF+46Kg24dER\nZeGcdratXlI0B6Aoq2Bw3bKa5t6M8sT1oiXcB6r6XeD3Sfvi1f6FwI1ZY4jI8cCxqrpDI+3lBmBF\nvPt84Evx668AL3BWBMMwjHrSSlH4zZ5L2qq6nHXAdwMkmfGTxm2f1lY0Rt/APgAe8soaQ1TAyM3B\nWROSuiqmkTn3cVKeuF40K6bgOcCQqt7tbZsfuw62i8hz4m0nAPd5x9wXb3P77gVQ1cPAfmBWY6dt\nGMZkpFX6C7TCXNKUkqyCP6EbIK26YDiu62kAURqhcxe4KoVhGqFfxMgnqfhR2blfckmJMvBA+yw6\ne3rHReZFtUxt0nVfRbGV4AFgrqruE5EzgM0i8rR6XUxELgMuA5g7t/n/1IZhjC/WrljUMqb6Zs/F\nzzjIiy/owx4Gbpwj8crb9/MnUa4T4crFc9mwY3ehk2JbUMI4l/slQbHo7OmN+ih4ZZ0nImNuKRCR\nqcDLgZvcNlV9RFX3xa9/CPQDC4H7gRO900+MtxH/PskbcyawL+maqvo5VT1TVc887rjj6ntDhmEY\nE5hyHQPzBOn59QEcvmVgY9+ewurfjesUCVeqOKlkcdI8AQbWLSucEyovme6XDFdB/1XnsSplzIlE\nM9wHLwR+qaoFt4CIHCcibfHrJwELgF+r6gPAQyLSFccLvBr4enzaLcBr4tevBL6tk63ogmEYRoPJ\nU+u/XIzD2hWLaBNBgQ1xWmHn7BkFAZvkEgl7GzhlYM3mnXT09DI/MOOH80hzZ7hxO2fPGFVmbr45\nNZDQV3ryVE0c7zQy++BG4BzgCcAQ8D5VvVZErgd2qKqfdvgK4APAo8CR+Nj/iPedCVwPTAduBd6i\nqioi04D1wDOIAhovVtVfl5uXZR8YhmHkp5y5PW82hDvOBQBW0pY46XwYtTwoUU2C/r0HcmdluIyD\nwauXl15r010AifMN77eaNstjTSXZB1bR0DAMw6iJStIkK1EygEIDI4iEvwS/HaFQLjunBMvA6W+/\niT8ePaMopVGIWiunKQGtmiLqU4lS0KxAQ8MwDGOC4AL7NuzYXVYwhjEDYffBI3F2gR8s6AcNOmVh\nY98eOmfP4O6hYRRSYwf8rIeNfXt4Y+dj+Ls3vKh0Yqq8NFBIkoR9GGjZ7MDPemOWAsMwDKMm/B4D\nWUWCHEkm97BQkS+Z/FbHaeeH+GmQbuXfnzS3SSADW6J4kWEYhlF/WjGwzc8OyENaYKFjSpBxMLBu\nWe7uhY61KxYVxuhft6xEIbj4Vf9Ep6fMGBHmPjAMwxhH+FH2rWK2rtSEnnT82hWL6BvYx66hYTpn\nz6jKLB8WSTr5uD9LtA4s/cjthaBEoxhTCgzDMMYR1RQPSqOSXgRjEVDnyhX37z1QuF7n7BlFAtyP\nNVi/Yzd9A/vYtnoJUFwkKSmroOPyLQC07T3QspkCzcbcB4ZhGOOIrHLClVCuF0FILT0Xyrk83H6/\ndsGGeG6utPH6HbtZ7ykEDtcJESKFafDq5SUKwfdfuqqQZiiUBiVWOt+JjFkKDMMwJiG+cPd7EaRZ\nIWqxUKS5PMLaA/3eCj5smZwVDth5xdbIgnHBqaU7VXkW8CzIrUi1ootmrDBLgWEYxgQnaeXrgvW6\nvaj+LCtEHgtF2gq7c/aMot8OZw2A4nTD8PwpcfMjgJnTpxY1QxKirIJQIejs6S1YByqlmqZTE8W6\nYEqBYRjGBCfN9D+iyoYdu2sSZL4wDK/j9t0dm/hdzIDDX/07C8H8nt4iK4ETzttWL2Fw3TKGD40U\n3Ar/+rUPMpAQO9DZ05s4j7z3WY2LptktreuFKQWGYRgTnKSVrxNeClULMj8uYf2O3SXXcYLSVSAc\niXsJOMJURtcYyd8fNl9yVQYHr17Oubu+XzyhuHlR2jw29u3JpSBUs+pvdkvremHFiwzDMCYhTqCH\nhYEqyTIICw4NxOl/SZkDYR+BtAwHl13guzUc83t6Ey0DHDkCIqlz97e7eWQVPvLvK2ke4w3rfZCB\nKQWGYUxE6tW4yAnEsOZ/2jWdqX/hnPZCamBSxcEwqNBXInJx881w0UWlc9h0V8WVDss9h7Aa4nhP\nX7SKhoZhGBOIPOZs34yfRF6ftwsGVCjrl3ctkaE4XiApsND56R0VLUdFShSCzp5eOi7fUnQ/aQGN\nPuXiBZzS4AIZx7s7oFJMKTAMw2hx8gh0CX6HhD7vNEXDF+5+Tr/fYCgpi8EXnn4RopCKSiKLlHQz\nvOr67XT29BbVNMhz3by4+3TpkePddVApphQYhmG0OHmC2FbFwnZVirANV8hpioZ/DQX6BvYVCgs5\nnGKQZorPmm+uyP777ktsbYwqX/jlcJHQBgrKTT2C/SZKwGC1WEyBYRhGhYxFyd9GXi8tyDDcH7Jw\nTnuhgqBzG5Tz4SddO/NeEpSBzp7e1FLMeeIIaprPBMBiCgzDMCqg0hS0sc5Jr+f1fIE/JRbASbEC\nzszvigZBZJZ321cunlvWJeHeL71me0ktg/U7dhdtT3IVsGlTSc2B0NKQZ2Wf9flOlPoC9cKUAsMw\nJj2VCoaxNjFXer1yQtAfN63gEEQFhbatXlKkCPhCee2KRXTOnlEQ8Bu8YEdfAXD9C8IyybuGhjly\nZCSxPPH8y7fAihVF954W7FjOHZH1+U52d0GIuQ8Mw5j0TDQTcpZJvVxnxDznuvoDnbNnFDUk8nGu\nibBegX/NpLbGrpMhwGCwv1pXQdLnO9E+8yysTkEGphQYhtFsGi2Qqh2/XKyBX9QnZOb0qbzstBMS\nSxSXzCEpiHDFCti0iaXXbGfX0DAzp09l+NBIwwR5rbEI4wmLKTAMw2hhGu3Hrra9spvPFJHEc7NM\n7MOHRopiEYCCK6HI5J+SVcCmTcBoOuH+g4dLntHaFYsKLo9aGw+Z2yAZUwoMwzDGmHp34SsXKJk3\nkNKlHbZPa2N+Ty8dPb1FQYNpSoxfz8ApJE45cD0P1l5waolCsGbTXVE3w6DugU/4PgxUzEO1sQiT\nEVMKDMMwxph6d+ErZ3nIa5nwV+nOSeDOCXsX+KRZFgD+4+YeBpP6FcSCPazC6FdJbEsYNwxUhPJK\nj2UY5MeUAsMwjHFAlnWhnOXB7e+cPSNTeKaN7XcndCmKC+e0F6onhue52IT+dctYNBBcK+5kCOlV\nGMsVP/LnAOWFfj2zNyY6DQs0FJHrgOXAXlVdFG+7EngD8GB82N+r6tZ43xXApcAI8FZVvS3efgZw\nPTAd2Aq8TVVVRI4GbgDOAPYBF6nqYLl5WaChYRgTjUqbHYXBdf75QOF138C+zOwC1xK5JCgxIW5g\nzdd+ysY77i2aowsq9JspVUO9AzcnWhBiqwQaXg+cm7D9GlU9Pf5xCsEpwMXA0+JzPiUibfHxnyZS\nJBbEP27MS4E/qOqTgWuAqxt1I4ZhGI2gXivStJVyOH7aitk/33dt+AqBiw9wqPe7cN3rrksNJNx4\nx70lcwx7FaQ9D3/7WMQHTOYgxIYpBar6XeD3OQ8/H/iyqj6iqgPAPcDZInI8cKyq7tDIpHEDsMI7\n50vx668ALxBJ+ms0DMNoTerl63bC64hqkbAMx0+L3m+f1lb02+HM8zOnTwVggecyKLm+CFx6adH2\njsu3sPQjtxeOCQVtuC3tefjbK3lmS6/ZTkdPb+6ARJhc9QuSaEZMwVtE5Kcicp2IPC7edgJwr3fM\nffG2E+LX4faic1T1MLAfmNXIiRuGYWRR6co/a0VayVguOK9o1Z4yfpJQ3X/wcNFvx7bVSxhct4zh\nQyOFyoQhg1cvL6lI+Ow3XV8oQuSsAEmr+XBbWutj/z4qWcW7+aa5QJJI6wYJkyPWYKyVgk8DTwJO\nBx4APjwWFxWRy0TkThG588EHHyx/gmEYRhUkCdwsQZJl9q5H6WU3PmR3EgwD98K5+0JaiVwJC/7v\nD6lZBc970RmpQYhJuOs44R22Pg5LK+d1Ffj3U4mi5siyWExUxlQpUNUhVR1R1SPA54Gz4133Ayd5\nh54Yb7s/fh1uLzpHRKYCM4kCDpOu+zlVPVNVzzzuuOPqdTuGYRhF5F2Z5yFt1ZxGHgXDpf6545ww\nXjx/FoPrlpUE+7nz/EZI3V3z6F+3jG9e0108355e1my6q3CeUx7yCG93HYG6+vK3rV5SSG/M+/z9\nAkxJNROy+jBMBBpa5lhEOoAtXvbB8ar6QPx6NbBYVS8WkacBG4mUhCcC3wIWqOqIiNwBvBXoI8o+\n+ISqbhWRNwOnquobReRi4OWqemG5OVn2gWEYY0k5H3Xa/jwR8Hn9335nRH88v2xxUknikvETwra+\n/jfv5fxPvT91XkDRGEn7wt4I9aRRMQLjKUOhJXofiMiNwDnAE4Ah4H3x+9OJLFCDwF97SsI/AK8H\nDgNvV9Vb4+1nMpqSeCvwljglcRqwHngGUUDjxar663LzMqXAMIxyNEqQJI2bJFzWbN7Jhh27UaA7\noQeBOyZJ0KddM0nw+p0MHYnXU4UpCYblHPIjvD//PTBuBGvIeApIrEQpmNqoSajqqxI2X5tx/AeB\nDyZsvxMoeeKqegj4y1rmaBiGkUSYolcNScI4HNcvChS6HJLM774g8s3hLusgaa5Ocdg1NFxSiTDt\n3t3vlYvnJrY1zqMMJN0/jLZrDq0Iecer1OLSKFx8w0TDKhoahmEE1CNP3SkAu4aGC4pAUgoelJYJ\nzlNPwB3jigjl8Zn7cQXO0hB2PXRCu3/dshKFoP/xJ7Bm0125/Ol+PIIf65AVNJg1blJJ5LRnY1SP\nKQWGYYx76h30VY9iOE5oL5zTXuSv98dNEv5ZK17/eDfWqpSgODdWiLMVZAnX/nXLSrZ39vRy/bXf\nYO2KRbkEcJ7SyuHnljVuUklk//zJXHConphSYBjGuKcVV4lOaG9bvaQiBSMrTz7rOuH4fswBUIjs\nX9U1r2SM7njb4NXLSxUC1UJWgSOtWFLSvPr3HsjdyClLsDvlZ8Gc9oIikFaJ0X8G1SqLEzW7oBym\nFBiGURdpQM5gAAAgAElEQVSa+SU6XleJScpMNXnySWWANwSWgFVd81KVk8suOy+15kDSddOKJSVR\nSSOnLAtNkpJR7nOvRVlsRUVzLDClwDCMutDML9F6176vlVBBSlOY0goOJeXJr9m8kyOxkHbb3bi+\nv31D/NoFKnZ3zWNw3bKSZ+NbB0568N6ifX7NgbR55lXE8gj6cumUaf0bsqwkruhStcrieFU0a6Wh\ndQpaEUtJNIzGMJ5StBpNVhpetal3SWO4bS7Y0JFUc6CEhEyE+e++hYGrX1qyPe2zbeRnHqZLVvLs\nxlMNgbGgVbokGoYxiWi11XqjyOMmCVeZ9Vh1JlXTcyvhVV3ziisOZn0O116bqBB0XL4FlWSRkKdR\nUb2ppcrhZF3l1wOzFBiGYVRAWrGhRq2kw4DB7q55BYGZdyXs5pGUVYAqp73/tkIzpKTiRW4OAoVA\nxXpUIsx6PmZ5qh8tUdGwVTGlwDCMclQqrNLM1WnbKxF4filiGF01h+enVS5cs3lncgGiX/2KNT//\nv5KKhmnjV1KJMO/9mZl/bDD3gWEYRg7SXAFZZvEkN0mauTqtqVHW+EmBdT5hgF0YbOiKJa3fsZsP\nXfft9IqECxeWKARurmlZEXnbF+d1K9Ri5i/nxrF0xOowS4FhGJOWeqzk6zG+/971PBBgwDP3+5aA\nXUPDBVO+73ufEhcL2jU0nJli6I8ZFjFKsxRUwtJrtrNraJiFc9pLOi/WC9+CkuTyCJ99NdaZiWLB\nMPdBBqYUGMbkJUsYA3XzYWc1IUrCF0L+yn3hnPaCcHX5+T6JAjwhiPCbq97GG09cWtKZ8O6h4cqz\nFnKQV6jWonylNYTym0k5xWntikUVCfqJFs9gSkEGphQYxuQlSzDUY3VYbRpdmnKSFmDocMrCysVz\nWXv+01I7GSbFA4SElomsOZYTlHm6PELtzzzJIhG2gy4XDDoZsJgCwzCMBPJW10vyKS+9ZjsdPb0s\nvWZ76vhZaXRJY7ptQGKToIVz2gGYOX1qYXXv7ADdXfMK1oO1F5xaqhDE5Yk7enoLQtKPB1g4p72o\nn0BY/riSvgQha1csYkqsgKQdn9YhshL69x4o+k08lniv/TlNhpTZWjFLgWEYRkDSCrajp7ewfzBl\nRZ21Gp3f01sSKxCuatOyHZL8/v1XnZfoKnikbSqn/N3Xi4omOVyRo6zVe9ozqHSlXe74elpmKln9\nT0aLQSWWgqmNnoxhGMZ4w/fTO3z/fhpupe/jhJATzf4yzF3niGpRYx8ff6XtuwuSFIKlH7l9dH88\nvq9QuGu7Md0+3/ee9gyS7i2LcscnPeNKqXROUGzxmCxKQSWYpcAwDMOj3ivJcLUOpZaGiuoiJCgD\nazbdlRhA6QcTLpzTXnidFJ/Q6pH29fpczFKQjVkKDMOYdGQJhqyVZCXZCmEGgp+JkGd+fgXB/qvO\ng7POggsSFjSqbIwVDzdndw+7hoaBSOAnpQb6loJaSwI3ujphvVb41VgXJhMWaGgYxqQjK2gureBQ\neJ7fmRBKAxH9YkIAi+fPKpjM04olrd+xuyBAITL3r9+xO7IOhBbOOJAwqRugH0yYVRzI9UoY8Loo\nVlu4J+uZ1qNHgvUzGBvMUmAYxqQjy5+dFNGedJ6fCw8UhL/77eMLRP+1syT4ZnxnIQASCxAtfPct\nXPQX81nL6Ep/19BwkUsiz2rYV0T8VXzadp+klX/WM21W/IBROWYpMAxjQpK14s1KT8taka5dsagg\n4BbEq3CXyucCEJMCEcPywJ2zZ5RYEhwCvOxn30mtSHjRX8wvWBv86IJKV/b+/fmKipujBNt9klb+\nlvI3MTClwDCMCUm1Jutyws2N27/3QFG64t1Dw3R3zSv47pNaGbuxk6wJECkUA1cv52NbPly8Q7VQ\noti/L7+2QDX32eYFLfpZBv1XnceqeP7lajrkoZEtlo36YkqBYRgTEiewjqiyZvPOzOJBlayyQ4G4\nITbhKxRW8H5BIiDzGq7Q0eDVy9n2jnOK9r349Z+k06uPEF5/7YpFBeUjSUCXuz83Vp66Bf547t6S\n3ApJ16tXPMCazTuZ39NLR09vQ5oVTeZGSA5TCgzDmJC4lbAT1kmr1UpXsEm+9KS6A2HcgPPRd8QC\nzeevn9JOf0IxpI7Lt3DP7Pl0zp5RtaAqd3/OKgClikulz8tlTCTtT+rqWM39uDgON5d6YxYNUwoM\nw5jAJLX7bZ/WVsgSyMo08AnbE4cFhaC4FLFfLvlISi0YZx3oed05Rds7Lt9Cx+VbAIpaIYdKRtr7\npPsvp1gkjZG0us9a8YfnplGL4A0VsHpjGQ4NVApE5DoR2SsiO71t/ywivxSRn4rIJhF5bLy9Q0QO\nisiP45/PeOecISJ3icg9IvJxkcgJJiJHi8hN8fY+Eelo1L0YhjE+8Veo7vX+g4eBKGLfZRjsGhrO\nXLlm9TRwY+w/eLggxJ0lwV/Z+gxevbzEOvDprr+E+BohSemG/nsXFBjeg7tn1yPBd2/4xyYJw6TY\nijwBmuVcEbUIXj9Oo9YUyiQsWLKBFQ1F5LnAMHCDqi6Kty0Fvq2qh0XkagBVvTwW6FvcccE4dwBv\nBfqArcDHVfVWEXkT8HRVfaOIXAxcoKoXlZuXVTQ0jMmN31lv8fxZJX0Fkkr+JnXjSypO5BcLCksM\nF1IGEyoSdvb0llQjbJ/Wxv6Dh4uumUTYQriklTLFxZAgWnG3UgXDvF0Vk6hHD4WJTktUNFTV74ar\nd1Xd5r3dAbwyawwROR44VlV3xO9vAFYAtwLnA1fGh34F+KSIiE62us2GYVSEL2CTVpcuBsEX0M4F\n4NcuSMpC8GMOfPN4d9e8RGWg4/ItRSmKfQP72LZ6SUVCMTTbJ1X+8+sPQKm1o1n4z9ePFajk/utR\nA8EYpZnFi14P3OS9ny8iPwb2A+9R1e8BJwD3ecfcF28j/n0vQGx52A/MAn7X6IkbhjG+CFf1fqGe\nJJwpPkuQJgkjv8BO38C+qKhQUr0Bon4FbfH5fhGitLmnlQhOsgwkCcmk47KeVZ5SzrXiP1/XwbFS\n4W5FjepLWfeBiPw/4MeqekBEVgHPBD6mqrszT4zO7SDBLSAi/wCcCbxcVVVEjgbaVXWfiJwBbAae\nBiwE1qnqC+PzngNcrqrL41iFc1X1vnhfP7BYVUuUAhG5DLgMYO7cuWfs3l126oZhTCCSmhK5Fbpv\n+neCye3PMsUDBRdEkuDsvGJrYlYBCd+5p73/NvYfPMzM6VP5yftenDj3Ss3j1fYb8Fs8TxFpqGl+\nMjYnagaVuA/yBBp+GviTiJwGvBPoB26oYXKvBZYDlzhTv6o+oqr74tc/jK+xELgfONE7/cR4G/Hv\nk+IxpwIzgX1J11TVz6nqmap65nHHHVft1A3DaAJ5A8nC4/z3LhjPxzf9b1u9hMF1yxhYtywy9cOo\nyyAIPNvgxQnsGhouykgo5PGLlCgEnT29rNl0V+Jchw+NAFGwYr1y/KuN8vdbPDc6Gt8C+1qPPErB\n4Vh4nw98UlX/FTimmouJyLnAu4GXqeqfvO3HiUhb/PpJwALg16r6APCQiHTFWQevBr4en3YL8Jr4\n9SuJAhgtnsAwJhh5hVtWqt7aFYuYkuDTd2mDvoAO6xuEpH3JdM6ewUVvXJFac8CfW9hMyRe6WTn+\nPm7eS6/ZXnPBIP8ZhBH+eeoLWNGfiUMepeBhEbkC6AZ6RWQKcFS5k0TkRuB/gJNF5D4RuRT4JJFC\n8c0g9fC5wE/jmIKvAG9U1d/H+94EfAG4h8iCcGu8/VpglojcA7wD6MlxL4ZhjDPyCrfwOL8GwZrN\nOwspha5zoBN6eXP0neDzz3dWBYBt7ziHRUP9RXNa+pHbiyoSOiUkCzfvcoLWb4/sd1h0VLIKDxWo\npPPCTo5p5xvjmzwxBX8OrAR+oKrfE5G5wDmqWrULoZlYSqJhjG/y+qF9XzyQ6hvPE1jnxxHAaHrh\nms07WXvBqSXXXvCuTTzaFq2durvmFY0ZxjcMrltWtM3NMYwlCO/bD570UyGr8f2nBWKGx/ipj/51\nmh0b0Ozrtzp1jSlQ1d8CXwWOjjf9DthU/fQMwzCqJ++q1F/th5ULl16zvVDV0F8Zp40d1jKY39PL\n6uXvTFQIUGX+Ex9XtMlfeYfxDWHMQ1qRonBubt7bVi/J7H+Qh6QiR0nHpF1nrGID0qwnZqmoH3ks\nBW8gitx/vKp2isgC4DOq+oKxmGC9MUuBYYxvqlkV+v0GBtctK3mftVIOrQRAYpphx+Vbigoflcsa\nqDSrYCxWw62+4k57Zq0+72ZT7+JFbwbOJqooiKreLSKza5ifYRhG1VSTl+5SDd1qfOGc9kKFQij2\nzy+c0160Gi9yGyQoAy+/5J/50YlPhfgafvBgVlGdSovujEU+fqvn/FuhosaTx1LQp6qLReR/VfUZ\ncfrfj1T16WMzxfpilgLDmPgklSBO85W740NrgF+q+HF/2s//fuKSkvM6Lt9Cd9e8kh4HAgwk1Sio\nI626Om7GvKzUcTb1rlOwXUT+HpguIi8C/h34j1omaBiG0UjCyHz3u28gsZRJIp2zZ7Cxbw+DVy8v\nUQhOu/IbzI87GULkMvDjBMYiN7pV/ejNmJd1N6wfeZSCHuBB4C7gr4maEr2nkZMyDMOohDAAzQkJ\nlz7oSCojDKWBhBClGIY1B76y6AV0XL6F/QcPF+oeuDS+VXF6okBRqmLaHGullQRhWChqrOdlRZDq\nRx73wQzgkKqOxO/bgKP94kPjCXMfGEbrUK86+2HgYNb+MEUQRkv7AlGp4SvPLZ3rprv48g/28OiI\nclSbcPFZxSWQKwkszNuHYLzg7s2VRp4o9zVRqLf74FvAdO/9dOA/q5mYYRgTj0pWwGEVPr9EcDVm\n50JZ4ZiwZuGazTuZ7ykELogwvM6COOBw8OrlJQpBx+Vb6Lh8C+t37ObRkUh1eHREC6tTiIRi5+wZ\nqQWPwhW0P4dWqQaYNI+8c3P3ptCSLg0jP3mUgmmqWrC5xa//rHFTMgxjPLEhFuwbEkzwIaGvXxgN\n6MsyO5fLTyceZ1Vgtt/Yt6fIv++CDcPrbHvHOalphlCqbLg5+XNwfRT8FXJapcA0BaGZJCkqGzyl\nLQt3b7XWSzCaTx6l4ICIPNO9ibsYHmzclAzDGE9o8LtcYyLf1+9W6FDqF/bPSxOcvvAJuyC6/RK8\nd9fpG9gXuRUSeiI460AWbi5ZykzavjQFoRz1siokjZOkqCilLaOzyOPbbxXLiJFMnpiCs4AvA78h\nUpj/HLgo7mY47rCYAsOoL2EKWuhbz/K159kHkdk/LaUwqURw1lzX79gdpQymWAZcO+Sk4EO/3XJY\nbrhSP3o152U9r0rGKxf/0Mi0QksfHHvqXeb4B8BTgL8B3gg8dbwqBIZh1J9wdRiufKtZSbt9Dmea\nBwoxCfN7euno6aVz9oyCNaBz9oxUv/hp77+N9Tt2s+X6t6UqBEBRH4GQztkz2LZ6SdH9JjUKyrMa\nrsZtkPW80sYrZxVIopHR/K2UNWGUkmopEJHnq+q3ReTlSftV9WsNnVmDMEuBYdROvVaS5cZJs0Ik\n4awJoXWhqPlQhjKQNp4rYFQYw2uGlNSQyD8+azVc79V4WqnmVlyZpzV3sqyFxlCJpSBLKXi/qr5P\nRL6YsFtV9fW1TLJZmFJgGLWR1S2vUipN00sSxD7dXfMSzf5JysDJ7/waj0x9TOrc/Dm1T2tj/8HD\nBdfCxr49HIl97s6eoIymOzpFxLVnHkvKdVeslEYI7DQXk6U0Noa6uA9ihWAKcKuqvi74GZcKgWEY\nteObp0MTsG+qzmNC97sXuvREX6hnjeGE8czpUQuXhXPaizr5LZzTzvJf/VeqdcApBOKN1901ryiC\n3gn44UMjDK5bxrbVS0qC8BQKr/3AwWYoBFBqnq/VFdCI7Ig0F5OlNDafPIGGd+bVMMYDZikwjHTy\n9AxIWjm6bUc8YQmUrFh9ge+vqttEis38sYk+bUXpyOyWl9DW2HcVtOVYkSbd69JrthcaJ21bvWTC\nm77H0tIw0Z9ls6iL+8AbbB3wO+Am4IDbrqq/r2WSzcKUAsNIJ8tnH1YK9IWj77v3V4C+gpE0bprf\n3ikMzmzvtoVjJrU4TrIMrHzVVXx/7qiSMHP6VF522gm5BFCoKDnFx83JhFc21cQ0mHJQX+pd0fAi\novbJ3wV+GP+YVDWMCUhaz4Ck4j3Op79raLjoPIczWzuFIG0MZ9p2ZnvfguAUAhjtT+DiCTpnzygS\nGLfc/rNEhWDNpruKFAKA/QcPlxTmCasthsWJXMElX7XxMw7GE2NZK6CabINWKeg0GcmTkjg/4edJ\nYzE5wzDGFifIXdqdE9RhpUCgoAA4X76vAIRxB26MwXXLGFy3rMiPn0RYdMixYcfuImWkUOZYhJ98\n7OKiY/3yxEm4IMER1aICSU74u/NCRcnFHTjGo+AaS6FbTUyDpS02jzzug2nAm4BnE/0ffQ/4jKoe\navz06o+5DwyjlEalGK7ZvJMNO3YXIvMh3fwfmpn9JkY+C+e0j/r033FOyf5vPvls3vCK9wJwVJsU\n+hWEuEh359YQSlseJzVXSrvX8cR4nrtROfWOKbgZeBjYEG9aCTxWVf+yplk2CVMKDKN8FcJ6kRaj\nkKciX6gUCFHjoruHhlGS0ww7e3pTYyKSmDl9apGLIi3g0TDGM/WOKVikqpeq6nfinzcAT6ttioZh\nNJPQfFxPc23Y6yCJpO1h10GXauhQIpfBwNXLSxSCjsu3sPQjtxdcHkk4V4iPrxBAFK/gxzZUw3ir\n7T/e5ms0lqnlD+FHItKlqjsARGQxFmhoGOMav1AQRAK5VjNymJa4sW8P/VedV1BAkgrT+L0IVsWR\n/H5tAH/lnmQZgMg6QNyl0N1L38C+orRBHz/GIMyc6N97gG2rl9T0LMLOiK3OeJuv0VjyWArOAL4v\nIoMiMgj8D3CWiNwlIj9t6OwMw2gIjaht74oPuQC+lYvnsmbzzoJCsKprXmJrYYisAOt37Kajp5f2\naW2FxkNZCsGaTXfRGfc+CK0cd3vBiH5GAUQuAWcNWDx/Vk0tf/P0FWj1lbgF9Rk+eWIKMm1oqpoY\n2isi1wHLgb2quije9niiegcdwCBwoar+Id53BXApMAK8VVVvi7efAVwPTAe2Am9TVRWRo4EbiJSW\nfUSdGwfL3bDFFBhGMmm18/Myv6e3EKiXp0Oiu2ZaN8IjqpmNi/xjw3K+aRkHeeeVlzzjtGLvAWNy\nUe8uibuzfjJOvR44N9jWA3xLVRcA34rfIyKnABcTxSqcC3xKRNricz4NvAFYEP+4MS8F/qCqTwau\nAa4uf7uGYaQRpuRVmq62qmteIY0wq0OiXw8g7Ro3rX9XLoXAv5ZvBk9LdXRllV3Ko0tHzEPSit8v\n05yGm9+RCq5Vb1rdWmG0DnncB1Whqt8FwqqH5wNfil9/CVjhbf+yqj6iqgPAPcDZInI8cKyq7tDI\npHFDcI4b6yvAC0QSep0axhgzXr+AQ8GWZE5es3knHT29zO/pLbm/tSsWMRCb5jf27Um8f2cZ8JUP\nHyFyFZx5/y+Kts+/fAtrNt1Vkg2wcE574Vq+ArLAK6LkF1RytQ3WrljElPjrIq/yk5Tb7+IY3G//\nPt3fwNoViwp1/ZtV08CKARl5aZhSkMIcVX0gfv1bYE78+gTgXu+4++JtJ8Svw+1F56jqYWA/MKsx\n0zaM/IzXL+C7va6DrrlPiDPLK1EhoSTlx93/+h27CwqAOzbNrA+RMhBaB05Z/e90XL6l4JbwryXx\nnP1CQy5mwQnpNhG2rV5SlEngqhAeiRWSvL50X+lwQj8pnsF/Bo3I7qiGZl/fGD+MtVJQIF75508o\nrgERuUxE7hSROx988MGxuKQxiWm1L+C8lgv/n9EXfKEg9o8Puxq6c5PGTqsf8OJffT+1k+GfHjO9\n8N4pGf6Y/ohJVRT97Aq/iuLGvj1FnQ3z4AdnOqHfv/dAYsBmvTsV1kqzr2+MH1JTEkXkYTKEtqoe\nW8X1hkTkeFV9IHYN7I233w+c5B13Yrzt/vh1uN0/5z4RmQrMJAo4TJrr54DPQRRoWMW8DSM39Ujv\nqyf+yh1InZsz+4cFjfxUtVVd8wophO4fSRhtjhQWAypHWgGiztkzILZczJw+leFDI2WLEvnKSNJn\nsMq7P6DodaWEKZ0hrfY3EGIVDY00Ui0FqnpMLPg/RhQQeAKRUL4c+GiV17sFeE38+jXA173tF4vI\n0SIynyig8I7Y1fCQiHTF8QKvDs5xY70S+LaWS6UwjEmIL7iyXBrhajLJ4uH7xwUKPQ1cP4I8CoGL\nGwgVgktfsYaOy7cU4g0cc46dBlAoZuQXNRIoad6UZhnx789/XWkMSKsI1FpiV8ari8toPHncBy9T\n1U+p6sOq+pCqfpooyC8TEbmRqKbBySJyn4hcCqwDXiQidwMvjN+jqj8DbgZ+DnwDeLOqjsRDvQn4\nAlHwYT9wa7z9WmCWiNwDvIM4k8EwjFGcAHOCM2/ufJbgcwGJvkvACeqj2tJjfQVof+RPqVkF33ry\n4sTzXEDi8KERBtctY/hQ9NXQJsLAumUlTZiSBF7W/VYqIP3jmxlUWotgbzUXl9E65FEKDojIJSLS\nJiJTROQS4EC5k1T1Vap6vKoepaonquq1qrpPVV+gqgtU9YWq+nvv+A+qaqeqnqyqt3rb71TVRfG+\nv3XWAFU9pKp/qapPVtWzVfXX1TwAw5jIJPm+/QyA9Tt2M7+nl44gmyBN4KzZvLNoFQ+Rr/+h2EJw\neEQTOwkCDFy9nJ0fvbBom+tkmMVRbVIkwMJ0wlDAJQm8LAFaqYD0j69UAfFJOq4SJaMWwW4xBkYa\neYoXdRC5EP4f0eLgv4G35ykU1IpY8SJjMuGv+GFUODr82AC/uE7Seb4QTMIvSeyaF+0aGk6MG/jl\n3Kfy1rd/uhCHMHxopFAeOYkwFTFvQSA/1mH40EhVJv8sq0lSmeY8BZv8ctD+cVboyGgE9S5eNKiq\n56vqE1T1OFVdMV4VAsOY6OQxk7tYgO6ueSUFh9z5QEEo+WmFbnVarlmQa16UllVw7qv+uSgOYUSV\nY6cnxz3PnD61bCnhtGfgX6PalXGWlSGpBkG5FbwbzykE/nFm1jeaTR5LwUKiqoJzVHWRiDydKM7g\nH8digvXGLAXGRCZcafrv/Yj5JOEYlhzu7prHhh27i1bv/ordrcKTSFMGyuGUjVraOvvHd86ekdoY\nKS/lAgsrDTxslUBFY/JQiaUgj1KwHfg74LOq+ox4207Xz2C8YUqBMZEJBU6aAEra7oSpw3cHAAXB\nWqkyAPkUAnfN0IXhejGEPRny3BtgAtiY9NRbKfiBqp4lIv/rKQU/VtXT6zDXMceUAmOiUU0jI18B\ncP5woKQQ0cI4LiDtvU+11oHwem5Fn2X1yNvUyDVpEmAgiEsoh63ojYlCXWMKgN+JSCdxPJKIvBJ4\nIPsUwzDGimoaGfk+a1eyeO2KRQyuW1aU8393bHp3JCkESTUH8mQVJHF33Oo4T1ZBHv+7Br/z4GIS\nXCyF5fIbk4k8loInEVUDfBbwB2AAuKRMh8SWxSwFxkTDtxTsGhouioRPOs6tfE97/21FxYZ8Aetb\nDEI3gmPrF9/CKXsHSrYnKQN+lkOI7/v3t1UafZ+0sq9mte8sEAJMiZ9J1rlmUTBanXq7D9pUdURE\nZgBTVPXhekyyWZhSYLQytQqY0KTuj+dnH0xJEfROMfADDBfOaefuoeHigMMKXAXdnoISBjO6/eHc\n0tL/smIF6pXOV+lnYGmERqtTiVKQ2vvAY0BEvgHcBHy7ppkZxgSmHitGP/2t0kh2oKTzn9/3wO8h\n4CsEM6dP5aGDh1MbHPkr+CRl4Olv+zIPTWsv2Z6Euyd3DT8dstyzC1MDw+dUrh9BEkmfWaV9C6q5\nrmG0KnliCp4C/CfwZiIF4ZMi8uzGTsswxh/1qCdfaZ66L/Td6t7v/OePs//g4cJ7IVqhu7LB5Xzu\nL7y7L9U6kKQQ+MWO1wctlv3YBRfPsH7H7oJCk4b/bNL6MjgBnbfscD0+M6sOaEwkyroPig4WeRxR\ndcNLVLWtYbNqIOY+MBpFvfPZ814z9P+H47vYgbDbYFbWgU81WQVJMQShed09jzANshYTvJ9V0Z0Q\nVxGS5ZLI+3lZTIHR6tQ1piAecAlwEXAucCdwk6p+taZZNglTCoxmUanACqmk5oC/zwn8NhHap7Ul\nBheGSsHM6VP5yZXnlsxh9bJ3sGnR8yuat2PhnPbElMmkUsHVEt5vJQpGWOgp7zgWU2C0OnVNSRSR\nQeDtwPeAU1X1wvGqEBhGM8nbwjgNv4mRT5b52r/OysVzS1obO3O7z8yRQ4kKQcflW4oUAj9VMcR1\nZXTdE119Azf/0J3Q3TWPKTnaH5fDjVVNqeCw0ZG/Pe95tczdMFqBTEuBiLQB/6CqHxi7KTUWsxQY\nzaQaU3NoZvcL8WQVLlqzeWchzsBZJjp6eovGDosRZbkKjmoTHh2J5pDWxMd3BQyuW1a0L2x4lNUI\nqNmr71pcAs2eu2GE1C37IE5FXA5MGKXAMJpJucj2JGHkKwRuRRp22nOCff2O3fQN7CtKIWwToW9g\nHx09vUWCHe+8JGXg4cdM59TV/15475/XOXtG4bUffe+UkKSsAncvvgKTNEbS+7Gm0gwEn2bP3TBq\nIU+dgmuAo4hSEg+47ar6o8ZOrTGYpcAYC6pdaSbFHeTpU1AL1ZYnHgysFdZrwDBak3oXL/pOwmZV\n1eqijZqMKQXGWFCtCTlvoFxS5H6l1NqrwCkF9TSXNyODwzAmOnUNNFTV5yX8jEuFwDDGirz1BsKg\ntDyBcr5gdK2GkziqTYreu2OTehUAoJo5XtI8YPReO2fPqDnAzq8bkBSw5/ZvCIIVDcOoD3myD+aI\nyLUicmv8/hQRubTxUzOM8UtWRoAv7MLiOXlWwnkL7vgxABDFG6RZBzrjAMRKsiJcFgFA/1Xn0b/3\nQCOftMEAACAASURBVK55ZUXnhxkA4Xhuv6u+WMl8LSvAMMqTp6Lh9cBtwBPj97uIUhQNw6gQ5x5w\nAi20KKSlHfq4Y8sd51Ouk6Fb5XfOnkGbSCGlMCTc4gvmvNaRLKXGV6bSqhb2X3VeVWmH9aheaBgT\nnTwxBT9Q1bNE5H9V9Rnxth+r6uljMsM6YzEFRjMpV8AoTBl0hIV98gYa3nrd3/LUBwdLtqfFDoRx\nAWEnRTeX8MqVFGOqJDCxnjEEFo9gTFbqGlMAHBCRWcTfAyLSBeyvYX6GMWnxew/4ONN2GAfgUCIr\ngjN9+ymBaQxevbxEIfCtAyFCtPL3zes/ed+LS+IMpogUVuqOPKtvd49AwRqQN4agkvHT3APWo8Aw\nypNHKXgHcAvQKSL/DdwAvKWhszKMCUCSkFq7YlHBJ+4LeSf8wjiAEOfH9wsOhSS5Cs782/W5Mws2\nxC4JN/++gX1FLgW30q7UjJ8VI1AuhqDa8Q3DqIy8vQ+mAicTLSZ+paqPNnpijcLcB0ajCM3TzsQv\nRKtrV7Snc/aMglB35vqwMmFY/S8PL971fT676Z9KtleSZujwqxE6qkmvrKTBUK3m/VZ3D7T6/IyJ\nS73rFPwl8A1VfVhE3gM8E/hHK15kTBbyfpmH+fp+meGQtOZAjqXXbC+yBoSVCEOqrTkQljl2dHfN\no29gH7uGhpk5fSr7Dx5ObFiU9Wys3G8x9jyMZlHvmII1sULwbOAFwLXAp2uY3Mki8mPv5yERebuI\nXCki93vbz/POuUJE7hGRX4nIi73tZ4jIXfG+j4skhEsbRo3kMUuv2byzYBXonD2Djp7eEoXAbyC0\na2i4aAXt3Azu9d2BoE5TCJJcBf/4vNcXFIKwaVGYUdC/9wBJbOzbU9g3fGik4PIIn0HWs6nE9D8Z\nsOdhjAfyKAUj8e9lwOdVtRd4TLUXVNVfqerpcfbCGcCfgE3x7mvcPlXdClFdBOBi4GlErZs/FTdq\ngkg5eQOwIP4pbe1mGDWS58vcCcUpIoWVd6gQbFu9pEhIu5gCJ1jX79hdSEks5zo4+tFHUq0DXzj7\n5QyuW0Z317wSK0BY8Cgpg0Hi4/z7dvd+JAhENEGXHwt0NMYDmQ2RYu4Xkc8CLwKuFpGjyadM5OEF\nQL+q7s5Y5J8PfFlVHwEGROQe4Oy4pfOxqroDQERuAFYAt9ZpbsYkJam7YLkv8qSmQDCavudW3eHK\nfP2O3RzVJoyMkJtyrgJh1KyfRLjduRBC94AT/n0D++jfe6BwLxv79hSOyXo2vhXBBKFhjA/yCPcL\niYoXvVhV/wg8Hvi7Ol3/YuBG7/1bROSnInKdiDwu3nYCcK93zH3xthPi1+F2w8hFWgrbxr49BaGe\nN5LdXwWuiiPyF85pL3QM9IsDhZTLOHCklScOYwec4G6f1la0XRgtjuRv27Z6Cd1d85gSKOZOqO8a\nGi5YL0KrQN7qhIZhjA/y9D74EzAIvERE3gIcr6rbar2wiDwGeBngerN+GngScDrwAPDhWq/hXesy\nEblTRO588MEH6zWsMc5J8oe72ABHVv+BUBiGefjOKjBFpFACOIwV8Jk5fWpJDIAjTRlICyZcuXhu\nSdGhBXPai+ojtImwKnYlbIiVhQ1ehUQn1F0qYnfXvBLzd97qhIZhjA/y9D54L/AlYBbwBOCLcRZC\nrbwE+JGqDgGo6pCqjqjqEeDzwNnxcfcDJ3nnnRhvuz9+HW4vQVU/p6pnquqZxx13XB2mbkwEklay\nTri1iTC4blmqQEsSon6znvk9vYXAw5WL53IkVjSybAIvO+2EkhiAcuWJ03BuCZ/+vQcKVoxQwLt5\n+fNzpYbDLAlfITJrgGFMLPK4Dy4BzlLV96nq+4AuoLsO134VnutARI739l0AuCXYLcDFInK0iMwn\nCii8Q1UfAB4Ska446+DVwNfrMC9jEpMWUBcSClHfwqDe9ikirF2xqGzgoDPtO/K6CmZOTw8LCt0S\nnbNnpKYPdnvKQlbDJiiNFWgFa8BYNzuy5krGRCWPUvAbYJr3/mhSVuR5EZEZRIGLX/M2fyhOL/wp\n8DxgNYCq/gy4Gfg58A3gzarqwrLeBHwBuAfox4IMjQpI61DoB9SlEQrRtMZE7dPaUvsZ+PjiuxJX\nwUOBi8DHd0W0eS6McqZ+/7kkWQJa0Tow1tUMrXqiMVFJLV4kIp8g+q6aC5wFfDN+/yKilfrLx2qS\n9cSKFxmOtAqEQFE5X78pUFLjn7TmRN1d83J3MYTqCxD5uAwCoFAh0WUWLPAKJrn9SVaDsay8V69r\njXW1QKtOaIwn6lLRUERek3Wiqn6pirk1HVMKjDTSvuj9lX5SNbosS0Eebrv2TZz8u9IVZ5pCkFaF\n0DG4bhnze3oLmQ9TREoq6bVKdb1WmYdhTGTqUtFQVb8UC/6bgB/GPzd52w1jXLNm8046enqZ39PL\nms07C4F1rmOfw/fbu1W2fy5EVoFqymkOXr28RCHICiRMKoKUhB/zsHLx3JIOiL4LoJn+8bFyRVgM\ngGHkI1UpEJGpIvIhovz/LxF1R7xXRD4kIkeN1QQNo1E4f7AfP5AUZ/BQguvAP9dZCcI8/yySAgmf\n9TfXlXUX7BoapqOnl8XzZzG4blmicjC/p5eFQfqhm5ubd1IMgd+1caJhMQCGkY+sQMN/JipUNF9V\nz1DVZwKdwGOBfxmLyRlGI3GrU/Feu5WrKza03qtOGLb09dnYt4cpOcJ2z+m/MzV24DfHzs49dzcX\nV3jIx6+gCFH6ZNaKPCklMy+1rsDHSljX2yJRD8uDWS+MViQrpuBuYKEGB8R9B36pqgvGYH51x2IK\njDz4bY+BQsljGK0W6BcHyhNUmBVImHT+zOlTeejg4dQui9tWLymaq8MFGxalOK5bljm3agPnsmIC\nwnbQYa2DtH31oNGBgPWIhbB4CmOsqFeXRA0VgnjjCNk1WAyjpcmzQnN+eCUSsK6QkVvZ+grBwjnt\nmQpBkqvgE39xUZGrYO2KRYU0R8fwofSGCL4lwK2Cu+N5DsRzdUqM64UQ4j+HausNZK3AwzLJSbUO\n+vceaEidg0ZbIOpheWjF1E7DyFIKfi4irw43isgq4JeNm5JhNJa8PvQ8bgMgNRPgMYcfTbUOfPi5\no/W/XFyAE8zuffu0tlTt259HKND9cst+y+NQGaqH4MxSJsIyyWNZ66DR49ejaFOrFH4yDJ8s98EJ\nRMWFDhJlHgCcCUwHLlDVmgoYNQtzHxhhCmFYeyDcv9DL71+7YlHBNH0ko8Vx3poDM6dP5Sfve3Hq\ntcNjhw+NpJrE/QqEMFprwZnR3T63ff2O3SV1DSzv3jAmHnWpU+AN9nzgafHbn6vqt2qcX1MxpcDw\nWyNDae2BtGJEAgx4vvml12xP7FWQRFZWgQts7N97oEjRcMqI2+bmmeQv95UJV5sgFPD+eb6C0H/V\neebfNowJTF2VgomGKQWTE18RcLECTniGwW5Jwt7R3TWvIFjDFX09KhI6fOtE38A+dg0NFywFoZIA\nxYpMUtXFkFCx8N9D7VYDq/hnGK1DvQINDaPlyJvGleQ/9wv6uLbBrsWxizHo6OnNrBa4Pu6OWK55\nUZ5OhkBinYGwT4ELKtx/8DAjnkKQ5KNPUwjKPTe/cJPrAFlLrEEr1QWw1D/DyE96izXDaEHCDn15\njgNK3AG+iTxp1Z+HalwFIXcHCogv7MOVe1b63toVi3I/j7BgUehSCOdRDX4sQ7PJ+zdTb8xaYoxH\nzFJgjCvyRpX7x4UC36Xo1bJ6rKSTYRa+qrJwTntBmELU6nj9jt2sj4sPLZ4/q6q5Qulzc+9d+WO/\nI2J3bEGZKJH1zUr9ayVriWHkxWIKjAmPaw4Eo0F9WS6CLOoZN+DwTf5+wJ9v3XD1C+odDJhWRMhW\nubVjz9BoFSymwJjQVLrKXxD77RfOaaf/qvOqUgi2fvEtDVUI3D11zp5RWNX68QYrF8+tacW7ZvNO\n5vf00hE3f3K4WIKwwJCtcmunlawlhpEXiykwxh15fcR+PQGI/PfzvTbIeWmEMgCRkhIKYVfhL41q\nBYwfaBk+t6TiTK0UE2AYxthhlgKjpQnbG0P+tr9O0LpofaWy+txJWQXP/6vPpCoEzh9frq2xK2Ts\nr87HosKfeK/DfWHmgq1yDWNyYjEFRkvj598n+dKTiu4k+cmB3BkGzxr8MRtvek/J9jzWgTAWwOFq\nI7g6B/4x1dQVcLiaCuWqHRqGMXmxmAJjXJK06vfbG7t2xuF+F0F/2vtvo6Ont1BLwK3EXQR/Hgav\nXl6iEFSSVZC20l8Q1/8PjxFG+xJAerxEmo/fxUe4GgZJPQ4MwzDyYkqB0TKEgs+tjru75jGwbllR\nQR/H2hWLmBILW79zYaUkuQo2nvbi3MqAMLriT3If+AqKf45C0T25wkEbAiUmj3vB73VgQYKGYVSD\nBRoaLUMY3BYKNxcweES10O536TXbE831eZk6cph7/mVFyfZKAwn9ngiL58/i7qHh1PgFdz9+yWV3\nz37VRUdWaptfdjkMHrQgQcMwKsViCowxo9K8bf/4JPN/d9e8qioROuqZVTC4bllJl0KHE/wOJ8iT\nehgkPaM8zYrqnRNvOfaGMXGwmAKjJanGrH0kMLn7VKsQJLkKoPY0Q3d/wmiGgWNhHFPgXAz9V53H\nqq55JVkUfQP7SsbN4zqot8vAXBCGMTkxpcAYMypNu/Nz631CgVsJ9SpPHLJm887C/a2KYyC6u+YB\nkZXA1R9wLo+Onl76BvYVtjkhHBYRgnzpgfVOaWxWaWDDMJpLU9wHIjIIPAyMAIdV9UwReTxwE9AB\nDAIXquof4uOvAC6Nj3+rqt4Wbz8DuB6YDmwF3qZlbsjcB61Hmql6zeadidaAo9qER0cq+7ttVAEi\nh1AcVwDp7Yw7vAJKg/E5aeWGDaMemDtocjNe3AfPU9XTvYn2AN9S1QXAt+L3iMgpwMXA04BzgU+J\nSFt8zqeBNwAL4p9zx3D+Rp3wO/aFJXiTqEQhaJSrIEQhtchSWIdgoVd2OWTx/FlWNMioO+YOMvLS\nSu6D84Evxa+/BKzwtn9ZVR9R1QHgHuBsETkeOFZVd8TWgRu8c4xxROfsGYXXfjpi5xVbaxq3Ea6C\nmdOzE3aU0XsIzf7unhbPn8XgumVsW72kcJ7/pV1pnYF61CWw2gYTG3MHGXlpllKgwH+KyA9F5LJ4\n2xxVfSB+/VtgTvz6BOBe79z74m0nxK/D7cY4w29Q5BSEpCj+vCRZBzre/R91sQ4MHxqhOw4QTIpt\nEEgtv5y1WvO/tCtZ1TkXS62rQFtJTmysbLWRl2YpBc9W1dOBlwBvFpHn+jvjlX/dgh1E5DIRuVNE\n7nzwwQfrNaxRB8KV6a6h4aprD9x44xXprgKpJTwxwglt11kwTDPs7prHFBH6BvYlCuqs1Zr/pV3J\nqi6pmVE12ErSMAxoUvEiVb0//r1XRDYBZwNDInK8qj4Quwb2xoffD5zknX5ivO3++HW4Pel6nwM+\nB1GgYT3vxaiNpJVpNa2NGxE34GoPuGDHEdVCyqA/bxcz4AIL/flXI2TXrliUe0XnF3yqZRVYyTUN\nw5i4jLmlQERmiMgx7jWwFNgJ3AK8Jj7sNcDX49e3ABeLyNEiMp8ooPCO2NXwkIh0iYgAr/bOMWpg\nLPzL7hp+PEE1JLkKXvK6j9fFVeCqJrZ5VgYn8H1h7xQEt9oOaxL4x9XbRG9mYcMw6kkzLAVzgE2R\nHGcqsFFVvyEiPwBuFpFLgd3AhQCq+jMRuRn4OXAYeLOqjsRjvYnRlMRb4x+jRnzhVS9hE6ZE+Xn5\n1XDGfT/nq//27pLt9cwqWL9jdyFN0M3TZQy45+KXEy632g7LOBuGYbQaY64UqOqvgdMStu8DXpBy\nzgeBDyZsvxOwJVKdaYTwClfJtfQraISrwI8X8OsIjKjSv/dAUT2Bziu2NtwtYBiG0QxaKSXRaBFq\nNUknuR+cm6Bz9oyqzedJroItJz+7rEJwVFv5IENfYXHZBc4NELoK3LEWsW8YxkTDuiQadccvRuRw\n5vfQXZCnOuGUIyP8+p/PL9me1zqQt9jREVVOe/9t7D94mIVz2ovqCECk7DgLx4gqC+e0F6oPGoZh\nTARMKTDqju+DD8sUhx0DywnssahG6FBg/8HDwKjyklZqGUb7GRiGYUwUTCkwqsIJSwFWBVH2/XsP\nlBzfJlKkLJQjSRmA+isEM6dPZfjQSEmMgwsozHINmIXAMIyJhsUUGFXhhKVSag1IEpZHKsg0qLU8\nsV+KOCueYOGcdoYPjbBy8dyifgSD65axeP6skpRJIYo3GFy3jMF1yyxo0DCMCYdZCoyqWLl4bqpZ\nPYk8Xv16uQqcCwBK3RN+HIAf+9DdNY/F82cVeg+4fX7mgTE5sI6CxmSmKa2Tm4m1Tq6ctC/Jpdds\nZ9fQMDOnT+Whg4errks9Vq4CXyFYu2JRSbxAmwgjqkWuDudeMAExeXCVKdtELGbEmBBU0jrZLAVG\nWcJiRms272TDjt0FJcBfmVfKWAUStonQv/dAUQqhHy8gQPu0NvYfPEz7tLZCXIS7N6c8mGLQmtRz\ndW9FpozJjMUUGGUJm+Vs7NtTc7eqpJoD8999S0OsA27uSZ0IIVIYVnXNKygA+w8eLhzrY/UIWpd6\n1oyw0tHGZMYsBUZZwkp8KxfPLbIUVMLnv/oBXnTPHSXb66EMzJw+NbXGgMNZOo7ECoHfn6BvYB+7\nhoZZOKe9cM++VcRWjq2Lre4Noz5YTEEL0iqBTv48gKI5ZeXvpzFWrgKXJpk2d/MZG4YxmagkpsCU\nghakmULLVwScSdaZ0Z25feGc9qL0QrdCTyNJGVjR/WF+/MST6zz7UfygQaDoeWYpXfVQyFpFqTMM\nw4DKlAKLKWhBQh/+WLJhx25GVNmwYzcrF89FiASqX9wnrDeQphAs+u09qdaBeigErrZASHfXvKJn\nGD7PLJ9xmm/a7+dQrrW09UQwDGO8YjEFLUgzu+mp99tvcVwpY+Eq2LZ6Seaq3L1PE95JpPmmk7o8\nprWWNv+2YRjjFbMUTFLSVruuQ2B31zyg8uC6pKyC2+ef0ZDYAdfi2Anheqzc06wIWZaHvGMYhmG0\nOhZTMEnx4xbCle3Gvj1MmRJVAywXL1BAlcEPvbRkc6OaFznyxAyYj98wjMmMBRpm0OpKQaMFmBu/\nc/aMknK/1TIWrgIBFsQVCTtnz+DuoWEUChYNyy4wDMNIxioajmPC6oH1xE8j3DU0XMjRd/n5lVJr\neeLcVghgikhq7QGgpI6C+fQNwzAqx5SCFqORAi30qTvFo14KQV5lQIiEfCXlkVcunptZN8GnmYGa\nhmEY4xlTClqMRgo0p3D4rgOIhHRe50E9XAUKFbkr2kSK3AJ5MgAMwzCMyrHsg0mEi4pfPH8WEJX1\n7bxiKwtS8v19krIKIL9C4Hz/leJH+VeSAWAYhmFUjlkKJhHO/H5EFWW0CFE590EjAglD68RRbVKS\n7eDaFjvlZeXiuSWBg85q0AxrgWU1GIYx0TBLwQQkrQaBC2KsxFUQKgRP+ruvV6wQCFGlRB9ltIPh\nwjntPDoSzWr40AiD65YxuG4Zw4dGGFFl19BwYp2BcvUHylUerBWrXGgYxkTDlIIJiBNW63fsLhKI\neU3tH97y4VTrwJEpbRXPR+MfCbb37z1A/1Xn0b/3QOIcnYvAb3/sU86F0GihbS4MwzAmGlanYIIQ\nRua71EO/oE+eroa1ugrCZkmO7q55Re2WnTBtZHEhM+8bhmG0ePEiETkJuAGYQ7SA/JyqfkxErgTe\nADwYH/r3qro1PucK4FJgBHirqt4Wbz8DuB6YDmwF3qZlbmiiKgVhwZ6wSFG5aP8kZeDCleu446TK\nhOngumWFa7dPa2P/wcMsnNPO/2/v3oPmqus7jr8/JARjHgxSkgyENDcSmIhtwAwTRcELjdyRSjG0\nRjK0AxZLBaU1irSowwiCZqR0RBQG5SIoQsiIlHCbiA6BXAwQ7gkkQBpClBEk5Rb49o/zO0/Os9nd\n53k2+zy7Z/N5zezk7G/POfv75Wxyvud3XXTWod3TEuf7mZnZwGv3yYu2AF+OiBWSdgWWS7ojfTY/\nIi4u7ixpGjAbeB+wF3CnpKkR8TbwA7JA4n6yoOBw4LZBKkdbKc5vUAwIeutEuM8fnuXOK07fJr2R\nfgNTxnTV7BAIWW2BJxUyM2tfLW8+kHQLcClwMPBqlaDgqwAR8e30/nbgPGAtcE9E7JfSTwI+GhGn\n1fu+Tq0pKMprDXrTzFEFfVmDoBN0YpnMrLP1p6agpR0NJU0ADiB70gc4Q9JDkq6U9N6UNhZ4rnDY\n8yltbNquTK/2PadKWiZp2aZNm6rtUlqz5i9mwrxbmTV/cXdv+8mjR3R30Kum2qiC5Xvt11BAUOwE\nWNnxrhN753dimczMci0LCiR1Ab8EzoyIV8iaAiYB04ENwHeb9V0RcXlEzIiIGaNGjWrWadtCca6B\nq5es6zGEb5umg4iatQOfnnPxNum92XlIth5BrWWCi0HCQA8PHCwecWBmnawlkxdJ2pksILg2Im4C\niIiNhc9/BOSPreuBcYXD905p69N2ZfoOJZ/0pzcDMQHRO+/0fF+5mFNxyubiFMVlrnb3ugpm1skG\nvaZAkoArgMci4nuF9D0Lux0P5I+UC4HZknaRNBGYAjwQERuAVyTNTOf8HHDLoBSiTZy7YFWvAcH2\nTk9cNHL40JrzBpy7YBXvpH4M1Z6i/YRtZtb+WlFTcDAwB3hY0sqU9jXgJEnTyYYprgVOA4iIRyT9\nHHiUbOTCF9LIA4DT2Tok8TY6eORBZQe3vsw70OzagVdff7t7VEGeH8ienq+7/1mCrYsXVfITtplZ\n+xv0oCAifsu2k9tBNqSw1jHnA+dXSV8G7BB3msoObvUCgoFaq6D4lF/ZVNDXJZ/de9/MrH15QaQ2\nV1kjkN98q6kWDMD2DTOsdfOuDAL6WhNQGUyYmVn78NoHbajYU79aAFBtDoJatQN9DQhGDu8ZH04d\n09XdVFBt1EC+DHN/b+zuW2Bm1r5aPnnRYCvD5EV5T/3K5YUr30P1YGCfsxewZUj/KoHmzBy/zXoJ\nxbwU06pxs4CZWXsqzeRFnayRcfn5MTulq1IZABTff+3uK2rWDvQ3IKhc2njy6BFMnHcrE+bd2j0R\nUm9P9o1O6tMp8xeYmXUCBwUDpJGbZD75UF+GGZ669OYeaf1pKsjlQwynjOnqEXA8ufHV7vf58sa9\nPf032izgGQLNzNqHg4IBMhBt59XmHJhz4jf7HAwMkZgzc3z3OgWvvLYFgKfqLJpUmf9aT/buY2Bm\nVn7uU9AmZs1fXHNFw3F/eoF7f/hP26T3t2YgX9b46iXrqvZPKJozc3zVG3xf+xiYmVl7aPelk63C\nuQtW1QwIGp1zQMBOEpNHj2DNi5t7LFJE+iwfVpiPZlAhvdYTf1/nIzAzs/JxTUGL5L31u941hJdT\nNX5RtWBg1ZjJHD33+1XPN3L40B7nmTqmi0VnHVrze4s3/r6mmZlZ+fSnpsBBwSDLb7bV5hrINVI7\nkFfnV6vez7+zWGvQ242+8jwOEszMyslDEttYPsKgmmodCfs6qiCvzp88ekSPP2FrD/98SeWrl6zr\n7ihYq+NgZQdAjxIwM+t87lMwiGbNX1w1vdHpiat1Blzz4uYef567YFX3REhTxnR1913IpxnOJyy6\nesm6HueqnLa4si+Baw7MzDqPg4JBUG9kQSNNBQI+W2N0QH7znjx6RHcTAGQdCBeddWiPm3l+rqD6\nClVFlUGC1zAwM+s8DgoGQbWAoNFRBZW1A5VP7PmrOFXyToVmgMqb+2dnjm9oNIFHIZiZdR53NBwE\nE+bd2r29vSsZVnYgLK6gWAwYXL1vZmbg0Qd1DWZQUHnTbrR2oKjYdFBsHgA8oZCZmW3Dow/awKz5\ni7sDgmqjCvb98k11A4I5M8czZ+b4bdr6g60TEOUjBKaO6UJkSyp7YSEzM2uUg4IB8uTGVzll6S01\nawfeGDqs6nF5MJDf+J+54Kju9QqmjunqMUwwX29g0VmHslNaz8BDBs3MrFHuaNhEf/2N27tnFexv\nU0G1ZoG8Z39l58Bq3PHPzMy2l4OCJskDgmrBwJwTv8m9Ew+sely1uQYaucH3JXAwMzOrx0FBE5y7\nYBVvvvwqa+efsM1nvfUbqHYj9w3ezMxawUFBE3zr+PfzrYq03kYVrL3gqIHLkJmZWQMcFGyPt96C\nYT07DH591ulcc0D9YYFzZo4fyFyZmZk1xEHB9rjmmh5vq9UOjBw+lAf/85ODlSMzM7OGlT4okHQ4\n8H1gCPDjiLhg0L587lw+9ui7eWZI1zYf1eovYGZm1q5KHRRIGgL8N/A3wPPAUkkLI+LRQcoA91z0\nmUH5KjMzs4FW9smLDgJWR8TTEfEmcD1wXIvzZGZmVkplDwrGAs8V3j+f0szMzKyfyh4U9ImkUyUt\nk7Rs06ZNrc6OmZlZWyp7ULAeGFd4v3dK6yEiLo+IGRExY9SoUYOWOTMzszIpe1CwFJgiaaKkYcBs\nYGGL82RmZlZKpR59EBFbJP0LcDvZkMQrI+KRFmfLzMyslEodFABExK+BX7c6H2ZmZmVX9uYDMzMz\naxIHBWZmZgY4KDAzM7NEEdHqPAwqSZuAdf08bA/gDwOQnXbgspWTy1ZOLls5lb1s4yOiT+Pxd7ig\noBGSlkXEjFbnYyC4bOXkspWTy1ZOnVy2Sm4+MDMzM8BBgZmZmSUOCvrm8lZnYAC5bOXkspWTy1ZO\nnVy2HtynwMzMzADXFJiZmVnioKAXkg6X9ISk1ZLmtTo/vZE0TtI9kh6V9IikL6b08yStl7QyvY4s\nHPPVVL4nJH2ykP4BSQ+nzy6RpFaUqUjS2pSnlZKWpbTdJd0h6an053sL+5eibJL2LVyblZJekXRm\nWa+bpCslvShpVSGtaddJ0i6Sbkjp90ua0OKyXSTpcUkPSbpZ0m4pfYKk1wrX77ISlq1pv8E2SCAp\nsAAAB41JREFULNsNhXKtlbQypZfqujVVRPhV40W2yNIaYBIwDHgQmNbqfPWS5z2BA9P2rsCTwDTg\nPODsKvtPS+XaBZiYyjskffYAMBMQcBtwRBuUby2wR0Xad4B5aXsecGEZy1bxu3sBGF/W6wYcAhwI\nrBqI6wScDlyWtmcDN7S4bLOAoWn7wkLZJhT3qzhPWcrWtN9gu5Wt4vPvAv9RxuvWzJdrCuo7CFgd\nEU9HxJvA9cBxLc5TXRGxISJWpO0/A48BY+scchxwfUS8ERHPAKuBgyTtCbwnIpZE9iv/KfCpAc5+\no44DfpK2f8LWfJa1bJ8A1kREvUm22rpsEfEb4KWK5GZep+K5bgQ+MVg1ItXKFhGLImJLersE2Lve\nOcpUtjpKf91yKQ8nAj+rd452LVszOSiobyzwXOH989S/wbaVVH11AHB/SjojVW9eWai6rVXGsWm7\nMr3VArhT0nJJp6a0MRGxIW2/AIxJ22UrW242Pf9z6oTrBs29Tt3HpJvxy8BfDEy2++0UsifI3MRU\nBb1Y0kdSWtnK1qzfYDuWDeAjwMaIeKqQ1gnXrd8cFHQoSV3AL4EzI+IV4AdkzSDTgQ1kVWVl9OGI\nmA4cAXxB0iHFD1P0XtohNZKGAccCv0hJnXLdeij7dapF0jnAFuDalLQB+Mv0m/0ScJ2k97Qqfw3q\nyN9ghZPoGYh3wnVriIOC+tYD4wrv905pbU3SzmQBwbURcRNARGyMiLcj4h3gR2RNI1C7jOvpWQXa\nFmWPiPXpzxeBm8nKsTFV6+XVey+m3UtVtuQIYEVEbITOuW5JM69T9zGShgIjgT8OWM77QNJc4Gjg\nH1LQQ6pa/2PaXk7W7j6VEpWtyb/BtipbIR9/C9yQp3XCdWuUg4L6lgJTJE1MT3CzgYUtzlNdqQ3r\nCuCxiPheIX3Pwm7HA3kP3IXA7NRzdiIwBXggVfO+ImlmOufngFsGpRA1SBohadd8m6xz1yqyMpyc\ndjuZrfksTdkKejyxdMJ1K2jmdSqe6wTg7vxG3AqSDgf+HTg2Iv6vkD5K0pC0PYmsbE+XrGzN/A22\nVdmSw4DHI6K7WaATrlvDWtnLsQwv4EiyHvxrgHNanZ8+5PfDZNWyDwEr0+tI4Grg4ZS+ENizcMw5\nqXxPUOipDswg+w9gDXApabKrFpZtEllv5weBR/LrQdZudxfwFHAnsHvZypbyNILsyWJkIa2U140s\nsNkAvEXW7vqPzbxOwLvImlhWk/UGn9Tisq0ma0/O/83lvdA/nX6rK4EVwDElLFvTfoPtVraUfhXw\n+Yp9S3XdmvnyjIZmZmYGuPnAzMzMEgcFZmZmBjgoMDMzs8RBgZmZmQEOCszMzCxxUGDWQSS9naZm\nXSXpF5LevR3n+qikX6XtY1VnlVBJu0k6vYHvOE/S2duRx70k3djo8WbWk4MCs87yWkRMj4j9gTeB\nzxc/VKbf/+4jYmFEXFBnl93IVokbNJKGRsT/RsQJg/m9Zp3MQYFZ57oX2EfZ2vBPSPop2aQr4yTN\nknSfpBWpRqELspn5JD0uaQXZ1K+k9LmSLk3bYyTdLOnB9PoQcAEwOdVSXJT2+zdJS9NCOt8onOsc\nSU9K+i2wb7WMS7pK0mWSlqV9jy7kY6Gku4G7UtlWpc+GSLo41ZI8JOmMlP4BZYvaLJd0e2Gq5X+V\n9Gja9/rm/tWbldPQVmfAzJovzb1+BPA/KWkKcHJELJG0B/B14LCI2CzpK8CXJH2HbG77j5PNynZD\nlVMDXAIsjojj01SwXcA8YP/IFpBB0qz0nQeRrTu/UNniVZvJpgufTvb/zwpgeY3vmZCOnwzcI2mf\nlH4g8FcR8ZKylUBzp6ZjpkfEFkm7K1sH5L+A4yJik6TPAOeTrWQ4D5gYEW9I2q3OX6fZDsNBgVln\nGS5pZdq+l2wdjL2AdRGxJKXPBKYBv8umb2cYcB+wH/BMpOVjJV1DdqOt9HGyOd+JiLeBl7V1Od3c\nrPT6fXrfRRYk7ArcHGl9AEn11hL5eWSL8Dwl6emUP4A7IuKlKvsfRja98JaUt5ck7Q/sD9yRyjqE\nbKpbyKbtvVbSAmBBnXyY7TAcFJh1ltfyp/VcuhluLiaR3VhPqtivx3HbScC3I+KHFd9xZj/OUTkH\ne/5+c+WOveTjkYj4YJXPjgIOAY4BzpH0/jygMNtRuU+B2Y5nCXBwXh2vbPXJqcDjwARJk9N+J9U4\n/i7gn9OxQySNBP5MVguQux04pdBXYayk0cBvgE9JGq5sxctj6uTz7yTtlPIziWzRnXruAE5LTSdI\n2j0dM0rSB1PazpLelzpbjouIe4CvkC1z29XL+c06noMCsx1MRGwC5gI/k/QQqekgIl4nay64NXU0\nfLHGKb4IfEzSw2T9AaZFtvb871Inv4siYhFwHXBf2u9GYNeIWEHWV+FB4Day5clreZZstbnbyFax\ne72Xov04HfOQpAeBv4+IN8mWsb0wpa0EPkTWjHBNytvvgUsi4k+9nN+s43mVRDNrO5KuAn4VEZ6D\nwGwQuabAzMzMANcUmJmZWeKaAjMzMwMcFJiZmVnioMDMzMwABwVmZmaWOCgwMzMzwEGBmZmZJf8P\noRrX+8UJp6UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1e433dfd390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "ax.scatter(RF.predict(X_test), y_test, s=4)\n",
    "ax.plot(y_test, y_test, color='red')\n",
    "ax.set_title('Random Forest: predictions vs. observed values (test data)')\n",
    "ax.set_xlabel('Predicted prices')\n",
    "ax.set_ylabel('Observed prices');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the best model to predict some diamond prices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Real price, Predicted price:\n",
      "1882, 1784.0\n",
      "9586, 9592.0\n",
      "5058, 4907.0\n",
      "2780, 2666.0\n",
      "2811, 2612.0\n",
      "644, 660.0\n",
      "1378, 1420.0\n",
      "552, 572.0\n",
      "7823, 7817.0\n",
      "12800, 13046.0\n"
     ]
    }
   ],
   "source": [
    "n_pred=10\n",
    "ind_pred = RF.predict(X_test[:n_pred,])\n",
    "print('Real price, Predicted price:')\n",
    "for i, pred in enumerate(ind_pred):\n",
    "    print(round(y_test.values[i]), round(pred), sep=', ')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
